Distributing video data in a system comprising co-located computers and remote human interfaces

ABSTRACT

System and method for operating a system including one or more co-located computers, e.g., rack-mounted blades, coupled to one or more remote human interfaces (RHIs), and where content or communication is received from sources, e.g., external content sources, internal or external communications sources, etc., and distributed to the RHIs. A first computing system generates first video signals corresponding to a first image for transmission to a display device in a first RHI. The video manager receives a second image from a source, and inserts second video signals corresponding to the second image with the first video signals to form third video signals. The display device of the first RHI displays a third image based on the third video signals, e.g., a picture-in-picture presentation of the second and first images. The first, second, and third video signals may include video streams and/or audio signals, and may be analog, digital, or both.

PRIORITY CLAIM

[0001] This application claims benefit of priority of U.S. provisionalapplication Serial No. 60/396,793 titled “DISTRIBUTING CONTENT IN ASYSTEM COMPRISING COLOCATED COMPUTERS AND REMOTE HUMAN INTERFACES” filedJul. 18, 2002, whose inventor is Barry Thornton which is herebyincorporated by reference in its entirety.

BACKGROUND OF THE INVENTION

[0002] 1. Field of the Invention

[0003] The present invention relates generally to computer systems andspecifically to transmitting video and/or audio signals between a humaninterface and a computer in a system of co-located computer systems overa transmission medium, for example, in a videoconferencing system.

[0004] 2. Description of the Related Art

[0005] The components of a computer system (such as PCs, minicomputersand mainframes) may be divided into two functional units—the computingsystem 102 and the human interface (or “HI”) to the computing system.For a PC (Person Computer), the computing system may be the CPU, memory,hard drive, power supply and similar components. The computing systemmay be comprised in a chassis which holds the motherboard, power supply,hard drive and the like. The human interface, on the other hand, maycomprise those devices that humans use to transfer information to and/orreceive information from the computing system. The most commonlyrecognized devices which form part of the human interface with thecomputing system include the monitor, keyboard, mouse and printer. Thehuman interface may comprise a variety of other devices, such as ajoystick, trackball, touchpad, microphone, speakers, and telephone, aswell as other devices too numerous to specifically mention.

[0006] In current computer systems, e.g., current PC architectures, thehuman interface (e.g., the display monitor, mouse, and keyboard, etc.)is closely located to the computer system, by a distance typically lessthan about 10 feet. The computing system 102 generates and/or receiveshuman interface signals, e.g., display monitor, mouse and keyboardformatted data, that are provided directly to/from the human interface130 or desktop via individual specialized cables as illustrated in priorart FIG. 1A. For example, for most PCs installed at workstations, thecomputer monitor 116, keyboard 112 and mouse 114 rest on the desktopwhile the computer chassis which holds the computing system 102 rests onthe floor underneath the desktop. Prior art FIG. 1B is a block diagramof the computer system illustrated in FIG. 1A. As indicated in FIG. 1B,the computing system 102 typically includes a processor 106, i.e., aCPU, a memory 104, and I/O interface logic, such as a video card 136 andan I/O interface card 137 which are coupled to the processor 106 throughan I/O bus 124. The computing system 102 also typically includes chipset logic 108 for interfacing the processor 106 and memory 104 with theI/O bus 124. As is well known, two or more computing systems 102 may beconnected together in a network configuration.

[0007] Many commercial businesses and enterprises make extensive use ofpersonal computers (PCs) in their daily operations. Typically, each userof a personal computer in the enterprise has a networked PC at theirdesk or work area. As the number of networked computer systems utilizedin an enterprise increases, the management of such resources becomesincreasingly complex and expensive. Some of the manageability issuesinvolved in maintaining a large number of networked computer systemsinclude ease of installation and deployment, the topology and physicallogistics of the network, asset management, scalability (the cost andeffort involved in increasing the number of units), troubleshootingnetwork or unit problems, support costs, software tracking andmanagement, as well as the simple issue of physical space, be it floorspace or room on the desktop. In addition, there are security issuesregarding physical assets, data protection, and software control. Inmany business establishments, such as call centers, there is no need forthe user to install software on his/her unit, and in fact, managementmay specifically forbid employees from doing so. However, the standardpersonal computer configuration inherently provides the user thisability because the system is typically located with the user, andincludes a floppy drive, CDROM, and one or more hard drives. Ensuringthat unauthorized software is not installed on any of the machines inthe network involves periodically personally auditing the softwarecontents of each machine, at substantial cost in time and effort.

[0008] In order to fully resolve the aforementioned issues, in somecurrent systems the entire computing system is physically separated fromthe human interface, specifically, by keeping the human interface(monitor, keyboard, mouse and printer) at the desktop or workstationwhile relocating the associated computing system (motherboard, powersupply, memory, disk drives, etc.) to a secured computer room whereplural computing systems are maintained. By securing the computingsystems in one room, the employer's control over the computer systems isgreatly enhanced. For example, since employees no longer have personalaccess, through the floppy or CD drive, to the memory subsystem,employees can not surreptitiously remove information from theircomputing system. Nor can the employee independently load software orother data files onto their computing system. Similarly, the employeecan no longer physically change settings or otherwise modify thehardware portion of the computer. Maintenance is also greatlyfacilitated by placement of all of the computing systems in a commonroom. For example, the repair technicians and their equipment can bestationed in the same room with all of the computing systems. Thus, atechnician could replace failed components or even swap out the entireunit without making repeated trips to the location of the malfunctioningmachine. Such a room can be provided with special HVAC and power systemsto ensure that the room is kept clean, cool and fully powered.

[0009] U.S. Pat. No. 6,012,101 titled “Computer Network Having CommonlyLocated Computer Systems”; U.S. Pat. No. 6,119,146 titled “ComputerNetwork Having Multiple Remotely Located Human Interfaces Sharing aCommon Computing System”; U.S. Pat. No. 6,038,616 titled “ComputerSystem With Remotely Located Interface Where Signals are Encoded at theComputer System, Transferred Through a 4-wire Cable, and Decoded at theInterface” disclose systems where a plurality of computing systems arelocated at one location, and the human interfaces associated with thesecomputing systems are remotely located at respective desktops.

[0010]FIG. 2 illustrates an exemplary prior art system where the humaninterface is remotely located from the computing system. The system ofFIG. 2 includes a computing system, an upstream encoder, a communicationmedium, a downstream decoder, and the human interface devices. Thedownstream decoder and the human interface devices are located remotelyfrom the upstream encoder and the computing system. This system employsa protocol wherein human interface signals generated by the computingsystem are encoded by the upstream encoder into a format which allowstransmission over a lengthy distance to the remote location where thehuman interface devices are located. The encoded signals are thentransmitted over the communication medium. The encoded human interfacesignals are received and decoded by the downstream decoder at the remotelocation, being converted back into the originally generated humaninterface signals for propagation to the human interface devices. Humaninterface signals generated by the human interface devices are similarlyencoded by the downstream decoder, transmitted over the communicationmedium, decoded by the upstream encoder, and provided to the computingsystem. Thus, to date the separation of the computing system from thehuman interface has involved extension of the human interface signals,(monitor, mouse, keyboard, USB (Universal Serial Bus) and other I/Osignals), i.e., extensions of already existing I/O signals, that is, thehuman interface signals are generated by the computer (or humaninterface device), are changed or reformatted as needed for transmissionto a distant or remote location, and then converted back to theiroriginal format.

[0011] In some enterprises, multiple channels or sources of informationmay be monitored by a user, such as, for example, telephone, television,video conferencing, audio, and/or web browser, among others. However,prior art systems which attempt to integrate such disparate forms ofinformation for presentation to the user, e.g., over an Ethernetnetwork, are unable to satisfactorily do so because of a number ofissues. These issues include one or more of bandwidth, protocol andhardware incompatibilities, and limited computation resources, amongothers.

[0012] For example, one approach for delivering video content to thedesktop, e.g., television content, includes installing a cabletelevision (CTVA) system at the desktop, including either putting atelevision set at the desktop or installing a TV card in the computer.However, CTVA systems generally require a complete independent wiringnetwork to each desktop which includes power splitters, line amplifiers,heavy cabling, and a master translator/re-modulator as a head endsource. This network can be quite expensive, unsightly, heavy, andlimited in the kinds of sources that can be fed over the system. Inaddition a TV set may be required which takes up valuable space andpower and may generate substantial heat. In the case of a TV card beingadded to the desktop personal computer, the associated hardware placesan additional load on the computer's performance, degrading its abilityto function as a computer for the purposes of the business.

[0013] Another approach for delivering video content to the desktopinvolves conversion of streaming video information into packet basednetwork data (e.g., Ethernet packets), and displaying the video usingthe computer system as a television set. However, using the Ethernetnetwork as a conduit for the content video has the dual degradingeffects of loading the network with non-computer data and, as above,tying up the computer's computation resources with non-computer-relatedtasks.

[0014] Furthermore, in returning information from the user's desktop(human interface or HI), prior art methods have typically either used asecond cable or USB to carry all the return information. A primarydisadvantage of deployment of the second cable is that standardenterprise desktop installations today typically already have a Category5, 6, or 7 communications cable connecting the desktop (the HI) to a‘back room’ where the return signals are destined. This second cableadds considerable cost in labor to the deployment of the computersystem, and effectively doubles the amount of space needed for thecabling. A primary disadvantage of the use of USB to carry the returninformation is that USB signaling for video and audio generallyintroduces considerable, and quite objectionable, delay or latency intothe signals. Typically such video and audio signals lack synchronizationbetween lip movements and the corresponding spoken words, resulting inlow and usually unacceptable quality. In addition, at USB frame-rates,the image stream takes on a ‘sequence of photos’ perception rather thana smooth and continuously flowing character.

[0015] Therefore, improved systems and methods are desired for addingcontent and/or communication distribution functionality to co-locatedcomputer systems, such as in a videoconferencing system.

SUMMARY OF THE INVENTION

[0016] The present invention comprises various embodiments of a systemcomprising one or more centrally located, e.g., co-located, computersand respective remote human interfaces, wherein the system supportsenhanced communications and/or content delivery and distribution.

[0017] In one embodiment the system may include a cabinet or chassis,referred to as a cage, which has a plurality of slots. The computers mayinclude computer cards (also referred to as “computer-on-a-card” or“blade”) which may be adapted to be inserted into respective slots ofthe cage. The cage may include a cage connector which is adapted tocouple to an edge connector on each inserted computer card. Eachcomputer card, referred to as a blade, may be operable to slide into aslot the computer cage, thereby engaging the computer card edgeconnector with the cage connector. Other forms of computers may also beused in the invention, the blade/cage form being but one exemplaryembodiment. Thus, in various embodiment, the co-located computers may beincluded in a common support structure, e.g., in one or more rack-mountstructures.

[0018] A “blade switch” may be operable to couple to each of the bladesin the cage. A “video switch” may be operable to couple to the bladeswitch, and may be operable to route video content to and/or from any ofthe blades. The system preferably includes a plurality of humaninterfaces located remotely from the co-located computers. Thecomponents of each human interface may include a keyboard, a pointingdevice such as a mouse, a display device such as a computer monitor,and/or any other human interface components. In one embodiment, each ofthe human interfaces corresponds to one of the computing systems. Eachcomputer may communicate with the human interface by sending andreceiving encoded human interface signals transmitted over one or moreconnecting cables.

[0019] Each computer may include all the elements that make up astandard personal computer, such as a PC motherboard with amicroprocessor CPU, memory, and network and interface logic, configuredon a single card, and may be referred to as a computer card. In oneembodiment, the network logic may include a LAN interface, Ethernet, orother network interface. In one embodiment, the interface logic mayinclude keyboard, mouse, video, audio, USB, and/or any other interfacecircuitry associated with a PC motherboard, as well as human interfacelogic. The computer card may also include one or more hard disk drivesor optical drives and a power supply which may be operable to convertthe local main power to the appropriate voltages for the computer. Thecomputer card may also include an interfacing edge connector, which maybe operable to transmit video, mouse, keyboard, USB, and Ethernetsignals, as well as any other peripheral or network signals to the userinterface or a network, respectively.

[0020] Thus, in a preferred embodiment, the system includes a pluralityof computing systems, where the plurality of computing systems arelocated at a common location, and a plurality of human interfaces, whereeach of the human interfaces is located remotely from the commonlocation, and where each of the human interfaces includes a displaydevice for displaying images. In one embodiment, at least one of theplurality of human interfaces includes a plurality of display devices,e.g., a plurality of computer monitors. In some embodiments, each of atleast a subset of the human interfaces includes a camera for acquiringan image, e.g., for use in a video conferencing system.

[0021] A plurality of communication mediums may couple each of theplurality of computing systems to at least one of the plurality of humaninterfaces. A video manager may couple to each of the plurality ofcomputing systems. The video manager may operate to selectively insertvideo data (analog or digital) from an external source into the videodata (analog or digital) output from the computing system for display.Thus, the display device may operate to display the first imageincluding the second image. In one embodiment, the first display devicemay be operable to display a PIP (Picture-In-Picture) image including atleast a portion of the first image and at least a portion of the secondimage, e.g., may display the video data using a PIP mechanism.

[0022] In a first embodiment, a first computing system may be operableto generate first video data for display on a display device in a firsthuman interface, where the first video data corresponds to a firstimage, and where the first video data is arranged in a scan line format.The video manager may be operable to receive second video datacorresponding to a second image from a source, e.g., a content orcommunications source, and insert the second video data into a locationin the first video data. In one embodiment, the first video data andsecond video data comprise digital data. As one example, the videomanager may operate to selectively access portions of the first videodata and second video data from different portions of memory, e.g., on ascan line basis, and provide the combined digital data for display. Forexample, the combined digital data may be transmitted for display.Alternatively, the combined digital data may be provided to D/Aconverters for conversion to analog video signals for display.

[0023] In a second embodiment, where the second video data comprisesanalog video signals, the second analog video signals are inserted “onthe fly” into the first analog video signals corresponding to the firstvideo data as the first analog video signals are output from thecomputing system. The combined first and second analog video signals(referred to as third analog video signals) are transmitted across thecommunication medium to the remote human interface. The display deviceof the first human interface may then operate to display a third imagebased on the third analog video signals, where the third image includesat least a portion of the first image and at least a portion of thesecond image, e.g., in PIP format.

[0024] In one embodiment, the video manager may be further operable to:modify format, protocol, size, and/or resolution of the second image fordisplay on the display device of the first human interface, optionallyin response to command signals specifying the display attributes of thedisplay device.

[0025] In one embodiment, the second image may be received from acontent source. For example, in one embodiment, the second image may bereceived from an external image source over a network, or from otherimage sources such as, for example, television sources (cable,satellite, broadcast, etc.), digital or analog video sources, etc.

[0026] As described above, in one embodiment, the first image mayinclude first analog video signals, where the second image received fromthe content source comprises second analog video signals in a scan lineformat. In this embodiment, the video manager may be operable to: storethe second image in the memory medium, receive the first analog videosignals from the first computing system, retrieve the stored secondimage from the memory medium in digital form, convert the second imagefrom digital form to the second analog video signals, and insert thesecond analog video signals into the first analog video signals on ascan line basis, thereby generating the third video signals.

[0027] In another embodiment, each of at least a subset of the pluralityof human interfaces may include an audio device. In this case the videomanager may be further operable to: receive audio signals correspondingto the first analog video signals, and transmit the audio signals withthe third analog video signals to the first human interface forpresentation to a user of the first human interface.

[0028] In other embodiments, the second image may be received from acommunication source. For example, each of at least a subset of thehuman interfaces may include a camera for acquiring an image. The secondimage may be received from a camera comprised in a second humaninterface of the plurality of human interfaces. In one embodiment, thesecond image may include a video-conferencing image of a user of anotherone of the computing systems. In yet another embodiment, the videomanager may be operable to selectively insert a plurality of secondimages into the first image generated by the first computing system fordisplay on the first display device, where, for example, the pluralityof second images include video-conferencing images of users of otherones of the computing systems. In one embodiment, the second image maybe received from an external camera over a network, for example, from avideoconference participant at an external station.

[0029] In one embodiment, the memory medium in which the second image(or plurality of second images) is stored comprises a frame buffer (alsoreferred to as a frame grabber), where the frame buffer represents orcorresponds to the screen of a target display device. For example, theremay be multiple second images stored in the frame buffer at locationscorresponding to their intended display locations on a target displaydevice. The images (e.g., on a scan line basis) may be read from theframe buffer when the corresponding portions of the first image arebeing transmitted, and inserted such that they replace the correspondingportions of the first image.

[0030] In one embodiment, the first computing system may be operable togenerate a sequence of first images to be displayed on the first displaydevice in the respective first human interface, where the video managermay be operable to selectively insert a sequence of second images intothe sequence of first images generated by the first computing system fordisplay on the first display device, where each second image of thesequence of second images may be inserted into a respective one of thesequence of first images.

[0031] It should be noted that in a preferred embodiment, the videomanager may be operable to selectively insert the second image(s) intothe first image without using any CPU cycles of the computing systems.In other words, in preferred embodiments of the present invention, theinsertion process may incur no extra load on the computer systems.

[0032] In one embodiment, the video manager may include a video switchcomprising a memory medium, and a processor coupled to the memorymedium. The video manager may also include a content processor coupledto the video switch through one or more video buses, where the videoswitch may be operable to couple to a plurality of co-located computingsystems, and to further couple to a plurality of human interfaces, whereeach of the human interfaces may be located remotely from the co-locatedcomputing systems, where each of the human interfaces includes a displaydevice for displaying images.

[0033] As described above, the video switch may be further operable to:receive a first image generated by a first computing system to bedisplayed on a first display device in a respective first humaninterface, where the first image comprises first analog video signals ina scan line format, receive a second image from the content processor,where the second image comprises second analog video signals in the scanline format, and selectively insert the second image into the firstimage generated by the first computing system for display on the firstdisplay device by inserting the second analog video signals into thefirst analog video signals.

[0034] In one embodiment, the video manager may be further operable to:select a first video bus from the one or more video buses, receive thesecond image from the content processor over the first video bus, andstore the second image in the memory medium, where, in selectivelyinserting a second image into the first image generated by the firstcomputing system for display on the first display device, the videomanager may be operable to retrieve the second image from the memorymedium and insert the second image into the first image.

[0035] In one embodiment, (either in addition to, or instead of, thecontent processor) the video switch may include a communicationsprocessor coupled to the video switch through one or more video buses.In this embodiment, the communications processor may be operable toreceive a first image from a camera comprised in a respective firsthuman interface.

[0036] The video switch may be further operable to: receive a secondimage generated by a first computing system to be displayed on a firstdisplay device in a respective second human interface, receive the firstimage from the communications processor, and selectively insert thefirst image into the second image generated by the first computingsystem for display on the first display device, as described above.Similarly, the video manager may be further operable to: select a firstvideo bus from the one or more video buses, receive the first image fromthe communications processor over the first video bus, and store thefirst image in the memory medium, wherein, in selectively inserting afirst image into the second image generated by the first computingsystem for display on the first display device, the video manager isoperable to retrieve the first image from the memory medium and insertthe first image into the second image.

[0037] In one embodiment, the system may comprise a videoconferencingsystem, where, in addition to a display device, each of the humaninterfaces includes a camera for acquiring an image, and where eachcomputing system is operable to generate a respective first image to bedisplayed on a display device in a respective human interface. The videomanager may be operable to: receive the respective first image from arespective one of the plurality of computer systems, receive one or morerespective second images from a respective one or more cameras of arespective one or more human interfaces of the plurality of humaninterfaces, selectively insert the respective second images into thefirst image, thereby generating a third image comprising at least aportion of the respective first image and at least a portion of each ofthe respective second images, and transmit the third image to the humaninterface for display on the display device.

[0038] In one embodiment, the video manager may be further coupled toone or more external cameras over a network, where each of the one ormore external cameras may be operable to acquire a respective fourthimage. The video manager may be further operable to: receive one or morerespective fourth images respectively from the one or more externalcameras, selectively insert the one or more respective fourth imagesinto the first image, thereby generating the third image comprising atleast a portion of the respective first image and at least a portion ofeach of the respective second images, and at least a portion of each ofthe respective fourth images, and transmit the third image to the humaninterface for display on the display device. Of course, as is wellknown, the images described above are preferably comprised in video datathat may also include audio signals or data that correspond to the videoimages.

[0039] Thus, the system may implement a videoconferencing system thatmay facilitate video/audio communications between a plurality ofvideoconference participants, including participants using humaninterfaces both internal and external to the system.

[0040] Thus, various embodiments of the present invention may providemeans for delivering content and/or communications to and/or from aplurality of remote human interfaces.

BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Other advantages and details of the invention will becomeapparent upon reading the following detailed description and uponreference to the accompanying drawings in which:

[0042]FIG. 1A illustrates a prior art computer system having a computingsystem and human interface devices directly coupled to the computingsystem through standard human interface cables;

[0043]FIG. 1B is a block diagram of the prior art computer system ofFIG. 1A;

[0044]FIG. 2 is a block diagram of a prior art computer system having acomputing system and one or more human interface devices remotelylocated from the computing system, where the computing system generateshuman interface signals that are encoded for transmission to the remotelocation and then converted back to human interface signals forprovision to the one or more human interface devices;

[0045]FIG. 3A illustrates a plurality of co-located computing systemscoupled to corresponding remote human interfaces through extenderdevices, according to one embodiment;

[0046]FIG. 3B illustrates the system of FIG. 3A, where each extenderdevice is included in a corresponding monitor, according to oneembodiment;

[0047]FIG. 4 illustrates a computer on a card and a cage for co-locatinga plurality of such computers, according to one embodiment;

[0048]FIG. 5 illustrates a computer system using first and secondextenders to communicate between a computing system on a card and aremote human interface, according to one embodiment;

[0049]FIGS. 6 and 7 are block diagrams of the computer system of FIG. 5,according to various embodiments;

[0050]FIG. 8A illustrates one embodiment of a system of co-locatedcomputers with communication distribution to a plurality of userinterfaces;

[0051]FIG. 8B illustrates one embodiment of a system of co-locatedcomputers with content distribution to a plurality of user interfaces;

[0052]FIG. 8C illustrates one embodiment of a system of co-locatedcomputers with both communication and content distribution to aplurality of user interfaces;

[0053]FIG. 9 is a block diagram of one embodiment of the system of FIG.8C;

[0054]FIG. 10 illustrates one embodiment of signal cabling for the videobuses of the co-located computers in the system of FIGS. 8A-8C;

[0055]FIG. 11 is a detailed diagram of a video switch for the system ofFIGS. 8A-8C, according to one embodiment;

[0056]FIG. 12 illustrates picture-in-picture insertion, according to oneembodiment;

[0057]FIG. 13 illustrates timing relationships between a plurality ofvideo bus signals, according to one embodiment;

[0058]FIG. 14 illustrates interleaved and progressive scanned videoframe formats, according to one embodiment;

[0059]FIG. 15 illustrates single and averaged pixel de-resolution,according to one embodiment;

[0060] FIGS. 16A-16E illustrate scaled inserted images in a computerdisplay;

[0061]FIG. 17 is a diagram of a content processor, according to oneembodiment;

[0062]FIG. 18 illustrates communication video sub-frames, according toone embodiment;

[0063]FIG. 19 illustrates signal spectrum allocation for extended USBsignaling, according to one embodiment;

[0064]FIGS. 20 and 21 illustrate processing of multi-media signals,according to one embodiment;

[0065]FIG. 22 is a block diagram illustrating video and audio signaldelivery to a video bus, according to one embodiment;

[0066]FIG. 23 illustrates content operations, according to oneembodiment;

[0067]FIG. 24 is a flowchart of one embodiment of a method for insertionof video signals for display at a human interface, according to oneembodiment; and

[0068]FIGS. 25 and 26 flowchart embodiments of methods for communicatinghuman interface signals between a computer and a remote human interface.

[0069] While the invention is susceptible to various modifications andalternative forms, specific embodiments thereof are shown by way ofexample in the drawings and will herein be described in detail. Itshould be understood, however, that the drawings and detaileddescription thereto are not intended to limit the invention to theparticular form disclosed, but on the contrary, the intention is tocover all modifications, equivalents and alternatives falling within thespirit and scope of the present invention as defined by the appendedclaims.

DETAILED DESCRIPTION OF THE EMBODIMENTS INCORPORATION BY REFERENCE

[0070] U.S. Provisional Patent No. 60/144,809 titled “A Technique ToExtend The Operating Distance Of A Universal Serial Bus” is herebyincorporated by reference in its entirety as though fully and completelyset forth herein.

[0071] U.S. Pat. No. 6,119,146 titled “Computer Network Having MultipleRemotely Located Human Interfaces Sharing A Common Computing System”,which was filed May 4, 1998, whose inventors are Barry Thornton, AndrewHeller, Daniel Barrett, and Charles Ely, is hereby incorporated byreference in its entirety as though fully and completely set forthherein.

[0072] U.S. Pat. No. 6,038,616 titled “Computer System With RemotelyLocated Interface Where Signals Are Encoded At The Computer System,Transferred Through A 4-Wire Cable, And Decoded At The Interface”, whichwas filed May 4, 1998, whose inventors are Barry Thornton, AndrewHeller, Daniel Barrett, and Charles Ely, is hereby incorporated byreference in its entirety as though fully and completely set forthherein.

[0073] U.S. Pat. No. 6,037,884 titled “Technique To Encode MultipleDigital Data Streams In Limited Bandwidth For Transmission In A SingleMedium”, which was filed Oct. 27, 1997, whose inventor is BarryThornton, is hereby incorporated by reference in its entirety as thoughfully and completely set forth herein.

[0074] U.S. Pat. No. 6,012,101 titled “Computer Network Having CommonlyLocated Computing Systems”, which was filed May 4, 1998, whose inventorsare Andrew Heller, Barry Thornton, Daniel Barrett, and Charles Ely, ishereby incorporated by reference in its entirety as though fully andcompletely set forth herein.

[0075] U.S. Pat. No. 5,926,172, titled “Video Data Transmission AndDisplay System And Associated Methods For Encoding/DecodingSynchronization Information And Video Data”, which was filed Sep. 23,1997, whose inventor is Williams Hanley, is hereby incorporated byreference in its entirety as though fully and completely set forthherein.

[0076] U.S. patent application Ser. No. 09/179,809 titled “A TechniqueTo Transfer Multiple Data Streams Over A Wire Or Wireless Medium” ishereby incorporated by reference in its entirety as though fully andcompletely set forth herein.

[0077] U.S. patent application Ser. No. 09/619,989 titled “System AndMethod For Providing A Remote Universal Serial Bus”, which was filedJul. 20, 2000, whose inventors are Dan Barrett, Mike Barron, and AndrewHeller, is hereby incorporated by reference in its entirety as thoughfully and completely set forth herein.

[0078] U.S. patent application Ser. No. 09/680,760 titled “System AndMethod For Combining Computer Video And Remote Universal Serial Bus InAn Extended Cable”, which was filed Oct. 6, 2000, whose inventor isBarry Thornton, is hereby incorporated by reference in its entirety asthough fully and completely set forth herein.

[0079] U.S. patent application Ser. No. 09/892,324 titled “ComputerSystem Having a Remotely Located Human Interface Using Computer I/O BusExtension”, which was filed Jun. 25, 2001, whose inventors are RayDuPont, Mike Tullis, and Barry Thornton, is hereby incorporated byreference in its entirety as though fully and completely set forthherein.

[0080] U.S. patent application Ser. No. 09/892,331 titled “SystemComprising Multiple Co-Located Computer Systems Each Having a RemotelyLocated Human Interface Using Computer I/O Bus Extension”, which wasfiled Jun. 25, 2001, whose inventors are Ray DuPont, Mike Tullis, andBarry Thornton, is hereby incorporated by reference in its entirety asthough fully and completely set forth herein.

[0081] U.S. Pat. No. 6,067,098 titled “Video/Graphics Controller WhichPerforms Pointer-Based Display List Video Refresh Operation”, which wasfiled Apr. 6, 1998, whose inventor is Thomas A. Dye, is herebyincorporated by reference in its entirety as though fully and completelyset forth herein.

[0082] FIGS. 3A-3B—Co-Located Computing Systems with Remote HumanInterfaces

[0083]FIGS. 3A and 3B illustrate embodiments of the invention where aplurality of computer cards 102A-102C are installed in respective slotsof cage 511, and where each computer card is be coupled via atransmission medium to a respective human interface, i.e., one or morehuman interface devices. It should be noted that in the systemsdescribed herein, the plurality of computing systems 102 may be a subsetof a greater plurality of computing systems included in the system,i.e., there may be other computing systems included in the system thatare not shown.

[0084] As shown in FIG. 3A, computer card 102A may be inserted into cageslot 512A, and may thereby be coupled to keyboard 112A, mouse 114A, andmonitor 116A, which include the human interface for that computer card.Computer cards 102B and 102C may be similarly inserted into respectiveslots 512B and 512C and coupled to respective human interfaces as shown.Thus, the computer cards 102A-102C may all be installed in the cage 511at a central location, while the user interface for each computer cardmay be located remotely from the cage 511, such as at the respectivework areas of the users of the computer cards. It should be noted thatthe human interface devices shown here are for illustration purposesonly, and that the actual type and number of devices included in eachhuman interface may vary.

[0085] As FIG. 3A also shows, in one embodiment, the one or more humaninterface devices which compose each human interface 130 may be coupledto a transmission medium through an extender device 121, such as a PCIor USB extender device. For example, the human interface associated withcomputer card 102A may be coupled to the transmission medium through theextender device 121A, as shown. In other words, the monitor 116A, thekeyboard 112A, and the mouse 114A (and any other human interface deviceincluded in the human interface for computer card 102A) may plug in tothe extender device 121A. Similarly, as FIG. 3A shows, the humaninterface devices corresponding to computer cards 102B and 102C may becoupled to their respective transmission mediums through respectiveextender devices 121B and 121C.

[0086]FIG. 3B illustrates a computer system similar to that describedwith reference to FIG. 3A, but where each extender device 121 isincluded in the monitor 116 of each respective human interface 130.Thus, as FIG. 3B shows, in one embodiment the monitor 116 of each humaninterface may provide ports for coupling the other human interfacedevices to the serial bus 110. As mentioned above, the inclusion of theextender device 121 in the monitor 116 is meant to be an illustrativeembodiment and is not intended to limit the invention thus. In otherwords, any of the human interface devices may be adapted to include theextender device 121, through which the other human interface devices maycouple to the serial bus 110.

[0087]FIG. 4—A Computing System On A Card

[0088]FIG. 4 illustrates the computing system of FIGS. 3A and 3B,according to one embodiment. As FIG. 4 shows, the computing system 102may include a motherboard 507 with CPU, memory, and networking logic, aswell as a power supply 510, and possibly a hard drive 508. Thus, thecomputing system 102 may comprise a “computer on a card”, also referredto as a “computer card” or “blade”. As shown, the computing system 102may further include an extender 120 which may operate to extend theoperational distance for a human interface located remotely from thecomputing system 102.

[0089] In one embodiment the computing system 102 may include a cabinet,referred to as a cage 511, having a plurality of slots 512. The computercard 102 may be operable to be inserted into a slot 512 of the cage 511,thereby making contact with a cage connector which may couple to thetransmission medium 110. Thus, the computer card may include a completePC on a single slide drawer frame which may be only 3 rack units high(5.25 inches), and thus may occupy a much smaller space than standard PCunits. The cage 511 may be operable to receive a plurality of suchcomputer cards via the plurality of slots 512, thus providing a meansfor co-locating a plurality of computing systems, each having a remotehuman interface, as described above. The cage may include a backplane orcommunication medium connecting each of the cage connectors, therebyenabling networking of the computer cards, such as in an Ethernetnetwork. Further details of the computer card may be found in U.S.patent application Ser. No. 09/728,667 titled “Computer on a Card with aRemote Human Interface”, and U.S. patent application Ser. No. 09/728,669titled “A System of Co-Located Computers in a Framework IncludingRemovable Function Modules for Adding Modular Functionality” which areboth incorporated by reference above.

[0090]FIG. 5A—Computer System with Remote Human Interface

[0091]FIG. 5 illustrates a computer system with a remote humaninterface, according to one embodiment. This embodiment is exemplary,and various other embodiments of the invention may be employed.

[0092] As FIG. 5 shows, the computer system may include a computingsystem 102 located at a first location, which is coupled to one or morehuman interface devices (collectively referred to as a human interface130) located at a second location through a transmission medium 110,e.g., serial bus or link 110. The second location is remotely locatedrelative to the first location. As used herein, the terms “first” and“second” are each used to refer to a location of a device at either thecomputing system location or at the human interface location. Thus a“first” device may be either at the computing system side or the humaninterface side, and similarly a “second” device may be either at thecomputing system side or the human interface side.

[0093] As FIG. 5 indicates, the computing system 102 may be a “computeron a card” or “blade”, i.e., the computing system 102 may be included ona circuit card which may include standard computing system componentssuch as a CPU, memory, and power supply. In one embodiment, thecomputing system 102 may further include an extender 120, which maycouple to the serial bus 110 and which may operate in conjunction withan extender device 121 at the remote location (the location of theremote human interface 130) to extend the functional distance betweenthe computing system 102 and the human interface 130. It is noted thatthe extender 120 may be a USB extender (e.g., using a USBX protocol), aPCI extender, or an extender for any other type of bus. Note that asused herein, the term “USBX” refers to an extension protocol whichallows USB based signals (i.e., based on any past, present, or futureUSB standards) to be transmitted distances greater than generallyallowed by the (past, present, or future) USB protocol. For furtherinformation on bus extension technologies, please see U.S. patentapplication Ser. No. 09/619,989 titled “System And Method For ProvidingA Remote Universal Serial Bus”, U.S. patent application Ser. No.09/892,324 titled “Computer System Having a Remotely Located HumanInterface Using Computer I/O Bus Extension”, and U.S. patent applicationSer. No. 09/892,331 titled “System Comprising Multiple Co-LocatedComputer Systems Each Having a Remotely Located Human Interface UsingComputer I/O Bus Extension”, which were incorporated by reference above.

[0094] In one embodiment, the one or more human interface devices may becoupled to the transmission medium 110 through the extender device 121,also located remotely from the computing system 102, which may beoperable to extend the functional distance between the computing system102 and the human interface. In one embodiment, the extender device 121may include an extender 120, e.g., an encoder/decoder and I/O devicehub, described in more detail below. In one embodiment, the humaninterface 130, i.e., the one or more human interface devices, may belocated more than 10 feet (or 20 feet) from the computing system 102.Thus, in various embodiments, the human interface may be located at adistance from the computing system 102 which is greater than typicallyallowed in prior art “non-extended” computer systems.

[0095] Thus, as used herein, the term “remotely located” is intended torefer to separation distances greater than those possible using currentconventionally designed cables such as those provided when purchasing aPC. Accordingly, the term “remotely located”, as used herein, generallyrefers to separation distances between 10 and 1,000 feet. However, as itis possible to utilize the disclosed techniques to separate thecomputing system 102 and the human interface 130 by distances greaterthan 1,000 feet, it should be clearly understood that the aforementionedupper limit of 1,000 feet is given by way of example and should not beconstrued as a limitation on the scope of the present invention. Theterm “remotely located” may also refer to a range greater than 15 feet,greater than 20 feet, etc.

[0096] The one or more human interface devices may include any of amonitor 116 a keyboard 112, a mouse 114, or any other human interfacedevice. Other examples of human interface devices contemplated mayinclude audio speakers (or headphones), a microphone, a printer, ascanner, a telephone, a removable storage medium, a biometric sensor, abarcode reader, a VR (Virtual Reality) interface device, and a PDA(Personal Digital Assistant) IR (Infra-Red) device, among others. Asmentioned above, the computing system 102 may be coupled to the one ormore human interface devices by a transmission medium 110. In apreferred embodiment the transmission medium may be a serial link or bus110. Various embodiments of the serial bus may include a 4-wiretransmission cable, e.g., cat5 cable, optical fiber, a wireless serialtransmission medium, a switched fabric bus, e.g., an Infiniband bus, anIEEE 1394 or IEEE 1394.2 bus, or any other serial transmission medium.In another embodiment, the transmission medium 110 may be a parallelbus.

[0097] FIGS. 6-7—Block Diagrams of a Computing System with Remote HumanInterface

[0098]FIGS. 6 and 7 are block diagrams of two embodiments of thecomputer system described above with reference to FIG. 5. As FIGS. 6 and7 show, the computing system 102, at a first location, may be coupledthrough a transmission medium, such as serial bus 110, to one or morehuman interface devices of a remote human interface 130, such askeyboard 112, mouse 114, and monitor 116, located at a second location.It should be noted that these embodiments are meant to be illustrativeand are not intended to limit the particular embodiments of theinvention.

[0099] In one embodiment, the computing system 102 may include a CPU orprocessor 106, a memory medium 104 coupled to the CPU 106, and an I/Obus 124 (e.g., a PCI bus or other local bus) coupled to the CPU 106, forexample, through chip set logic 108. As FIGS. 6 and 7 show, in oneembodiment, the computing system 102 may also include an extender device121A, e.g., a host extender device 121A, coupled to the I/O bus 124, asindicated. The host extender device 121A may operate to receive I/O andvideo signals from the computer 102 and convert the signals into a formsuitable for transmission over the serial link 110 to the remote humaninterface 130, as described in more detail below. In an embodiment inwhich the computing system 102 is a computer card 102A (i.e., a circuitcard), as described above with reference to FIGS. 3A-5, the extenderdevice 121A may be included on the circuit card or on a module coupledto the circuit card.

[0100] As FIGS. 6 and 7 also show, in one embodiment, the computersystem may also include an extender device 121B, e.g., a remote extenderdevice 121B, coupled to the one or more human interface devices, wherethe extender device 121B may be remotely located relative to the firstlocation, i.e., remotely located relative to the computing system 102.The remote extender device 121B may operate to receive encoded signalsfrom the host extender device 121A and decode or convert the signals tovideo signals and I/O device signals for provision to the monitor 116and one or more I/O devices, respectively. Similarly, the remoteextender 121B may operate to receive I/O signals from the I/O devicesand convert the I/O signals into a form suitable for transmission overthe serial link 110 to the computer 102, e.g., to the host extenderdevice 121A. The host extender device 121A may then operate to convertthe received encoded signals back to I/O signals and transmit the I/Osignals to the computer 102, e.g., via the local I/O bus 124. Furtherdetails of the host and remote extender devices 121 are provided below.It should be noted that the extender devices 121A and 121B may be any ofa variety of extender devices, e.g., may be implemented in accordancewith any of the methods described in the U.S. Patents and U.S. PatentApplications incorporated by reference above.

[0101] In a preferred embodiment, the host extender device 121A mayinclude a USBX controller 121A, and the remote extender device 121B mayinclude a USB remote root hub 121B, both of which are operable toconvert between USB signals and a USB extension protocol, e.g., USBX,where the USBX protocol allows the transmission of USB signals overdistances substantially greater than generally allowed by the USBprotocol, as described in U.S. patent application Ser. No. 09/619,989titled “System And Method For Providing A Remote Universal Serial Bus”,incorporated by reference above. It should be noted, however, that otherbuses and protocols may also be used to communicate the I/O and videosignals between the host computer 102 and the remote human interface. Inone embodiment, the remote extender device 121B may include a C/Port, asprovided by ClearCube Technologies, Inc., which operates toencode/decode HI signals for the various HI devices of the remote humaninterface 130, as described in one or more of the U.S. Patents andPatent Applications incorporated by reference above.

[0102] As shown in FIG. 7, in one embodiment, the host extender device121A may include extension technology for converting and transmittingI/O signals and video signals to the human interface devices at theremote location. For example, in one embodiment, the extender device121A may include a USB host extender 120A which may be operable toreceive I/O signals from the computer 102, e.g., USB signals, andconvert the signals into a form suitable for transmission over theserial link 110, e.g., USBX signals. Similarly, in one embodiment, theextender device 121A may also include a video extender 123 which may beoperable to receive video signals from the computer and convert thevideo signals into a form suitable for transmission to the humaninterface over the serial link 110, e.g., USBX video signals, as shown.In one embodiment, the video extender 123 may couple to a video card 136which may in turn couple to the I/O bus 124 of the computer 102. In analternative embodiment, the video card 136 may be included in theextender device 121A.

[0103] Thus, in one embodiment, the extender device 121A, e.g., the USBhost extender 120A and/or the video extender 123, may transform signals,e.g., USB signals, received from the computer 102 into a specialextension protocol, such as, for example, the USBX protocol, and thevideo extender 123 may receive video signals from the video card 136,and may convert the signals into a form suitable for transmission overthe serial link 110, to the remote human interface 130, such as, forexample, into USBX video signals.

[0104] Similarly, the extender device 121A, e.g., the USB host extender120A, may operate to receive signals (e.g., USBX signals) from theremote HI 130, e.g., the remote extender device 121B, over the seriallink 110, and convert the signals for provision to the computer 102,e.g., may convert the USBX signals to USB signals, which may then betransmitted to the computer 102.

[0105] Thus, in one embodiment, the host extender device 121A mayinclude a host extender 120A and a video extender 123 for converting orencoding and/or decoding HI signals for communication with the remoteHI.

[0106] As FIG. 7 also shows, in one embodiment, the remote extenderdevice 121B, i.e., the extender device 121B at the remote humaninterface, may include a video decoder 125, coupled to the serial link110 and to the computer monitor 116. The video decoder 125 may beoperable to receive the encoded video signals over the serial link 110,e.g., USBX video signals, and convert the video signals into a formsuitable for display by the video monitor 116, e.g., into RGB signals.In the embodiment shown, the remote extender device 121B may alsoinclude a remote extender 120B, e.g., an HI encoder/decoder 120B, andI/O device interface circuitry such as a USB hub 126, coupled to theserial link 110 and to one or more I/O devices, e.g., USB devices suchas, for example, keyboard 112 and mouse 114, as well as any other kindof I/O device or peripheral desired, including, for example, audiospeakers, 117 IP telephones 119, microphones 118, PDA IR elements 111,headphones, printers, bar-code readers, and so on. In a preferredembodiment, the human interface devices are USB devices. It is notedthat the one or more human interface devices may be coupled to the humaninterface circuitry 126 in a number of different ways, includingstandard interface cables, USB, wireless media, e.g., as specified bythe 802.11 protocol, optical fiber, or any other suitable communicationmedium. The remote extender 120B may operate to receive I/O signals,e.g., USBX signals, from the host extender device 121A, convert thesignals into signals suitable for propagation to the I/O devices, e.g.,into USB signals, and send the converted signals to the USB hub 126, asshown. The USB hub 126 may then propagate the signals to the respectiveI/O devices.

[0107] Thus, in one embodiment, each computer card may include an I/Obus 124 and a host extender device 121A, and each corresponding humaninterface 130 may include a remote extender device 121B which includescircuitry for receiving video and I/O signals over the serial link 110,converting the signals into an appropriate format, and propagating theconverted signals to respective human interface devices, where the firstextender device 121A and the second extender device 121B may be coupledvia the transmission medium 110, e.g., the serial bus.

[0108] In one embodiment, the remote extender 120B may also operate toreceive I/O signals from one or more of the I/O devices, and encode theI/O signals into a form suitable for transmission over the serial link110 to the host computer 102, e.g., USBX signals. For example, in oneembodiment, the USB hub 120B may be operable to receive USB signals fromvarious of the I/O devices, e.g., mouse 114 and keyboard 112, andconvert the USB signals into USBX signals for propagation over theserial bus 110 to the USB host extender 120A included in the extenderdevice 121A of the host computer system 102, which, as described above,may operate to convert the USBX signals back to USB I/O signals and sendthe I/O signals to the computer 102.

[0109] In one embodiment, the extender device 121B may be included inthe display device or monitor 116. The other human interface devices maythen be coupled to the serial bus 110 through the monitor 116, i.e.,through the extender device 121B included in the monitor. Note that inthis embodiment, the monitor 116 may include the extender device 121B,which itself may include remote extender 120B, USB hub 126, and videodecoder 125. The one or more human interface devices may be coupled tothe monitor in a number of different ways, including standard interfacecables, wireless media, e.g., as specified by the 802.11 protocol,optical fiber, or any other suitable communication medium. It is alsocontemplated that in other embodiments, the extender device 121B may beincluded in any of the human interface devices, which may then functionas a human interface hub for other human interface devices.

[0110] It should be noted that in the preferred embodiment, the one ormore human interface devices operate as if they were located in thefirst location and directly connected by human interface cables to thecomputing system. In other words, the extension of the human interfacemay be transparent to the user.

[0111] In another embodiment, the extender device 121B may not includespecial interface circuitry, e.g., the video decoder 125 and the I/Odevice interface circuitry (the USB hub) 120B shown in FIG. 6B. In thisembodiment (not shown), the serial link 110 may couple through anextender 120, e.g., a split bridge, to a local bus included in theextender device 121B, e.g., a PCI bus, which may then couple to thevarious 1/0 devices.

[0112]FIG. 8A—A Communications Distribution System

[0113]FIG. 8A is a high level diagram of a communications distributionsystem, according to one embodiment. As used herein, the term“communications” refers to any type of communication signals transmittedbetween users of the system, especially video/audio signals transmittedbetween human interfaces 130 in the system, such as, for example, videoconferencing signals generated by cameras and/or microphones at users'work areas, and/or telephone communications. In other words,communication signals may include any signals that provide verbal/audialand/or visual connectivity between two or more individuals.

[0114] As FIG. 8A shows, in one embodiment, the communicationsdistribution system may include a co-located computer system 802, e.g.,a plurality of computing systems, e.g., blade computers 102, that may becoupled to a plurality of human interfaces 130 through a video manager800A. In the embodiment shown, the computers (blades) 102 are includedin a plurality of cages 511. For example, in one embodiment, the cages511 may be rack-mounted in one or more component racks, as is well knownin the art. As noted above, in the systems described herein, theplurality of computing systems 102 may be a subset of a greaterplurality of computing systems included in the system, i.e., there maybe other computing systems included in the system that are not shown.

[0115] As FIG. 8A also shows, in one embodiment, the co-located computersystem 802 may be coupled to a second co-located computer system 803that includes a cluster computer system 803. The cluster computer system803 may similarly include a plurality of cages 511 containing respectivepluralities of blade computers 102. Alternatively, the cluster computersystem 803 may include the blade computers 102 in a single cage 511, orin yet another embodiment, without a cage 511. The cluster computersystem 803 may provide additional computation resources for the contentand communications distribution system 800. For example, the clustercomputer system 803 may provide burst capabilities, where surges incomputation loads of the co-located computer system 802 that exceed thecapacity of the co-located computer system 802 may be off-loaded to thecluster computer system 803. As another example, when a blade computer102 in the co-located computer system 802 fails, the cluster computersystem 803 may provide temporary blade functionality until the failedblade computer is replaced. As yet another example, the cluster computersystem 803 may provide various server functions for users, or othersub-systems, of the content and communications distribution system 800.For more detailed information regarding co-located computer systems,please see U.S. Pat. No. 6,012,101 titled “Computer Network HavingCommonly Located Computing Systems”, filed May 4, 1998, which wasincorporated by reference above.

[0116] The co-located computer system 802 may also couple to networkswitches 812 which may provide access by the co-located computer system802 to networks, e.g., LAN, WAN, the Internet, etc 840A, to facilitatecontent or communication file transfers and swapping, e.g., email, FTP,messaging, chat, etc.

[0117] As described above with reference to FIGS. 3-7D, each humaninterface 130 may be at a remote location from the co-located computersystems 802, and may include any of various human interface devices,such as, for example, any of one or more monitors 116, a keyboard 112, amouse 114, or any other human interface device. Other examples of humaninterface devices contemplated may include a joystick, trackball, audiospeakers (or headphones), a microphone, a printer, a scanner, atelephone, a removable storage medium, a biometric sensor, a barcodereader, a VR (Virtual Reality) interface device, and a PDA (PersonalDigital Assistant) IR (Infra-Red) device, among others. As FIG. 8A alsoshows, in a preferred embodiment, one or more of the human interfacesmay include a camera 135 which may be used for video conferencing,telepresence applications, videophones, etc.

[0118] In one embodiment, one or more of the human interfaces mayinclude multiple monitors, as shown. In other words, images transmittedto the human interface 130 for display may be distributed across aplurality of computer monitors 116. For example, in one embodiment, oneof the plurality of monitors may be used specifically forteleconferencing, where images of other users who are participating inthe video conference are displayed on the designated monitor. As anotherexample, an image (or image stream) for each participant may be shown ona respective monitor, e.g., adjacent to other visual informationassociated with that participant. Of course, these are merely examples,and are not intended to limit the use of multiple monitors to anyparticular configuration or approach.

[0119] As shown, in one embodiment, the video manager 800A may include ablade switch 809, a video switch 808, and a communications processor804. The video switch 808 may couple to the communications processor806, described in more detail below. The communication processor 806 maycouple to a video bus 810, and may also couple to one or morecommunication networks or transmission media, e.g., a TELCO network(telephone company switched circuit network) 850 and/or LAN, WAN,Internet 840B etc.

[0120] The video bus 810 may provide for high quality video streaming,e.g., raw analog RGB signals, for example, between the system and, say,other communications systems located in the same building or in otherbuildings on the same campus, e.g., within the enterprise.

[0121] The LAN,WAN 840B may provide for communications via a packetprotocol 807, such as, for example Ethernet. The LAN,WAN 840B may beused for IP based communications, e.g., IP based streaming video(quality of service), operating in an isochronous mode, whereinformation (packets) is transmitted as fast as possible, e.g., withoutregard for error-checking, collisions, etc. The LAN,WAN 840B may beparticularly useful for communication between the system and distantlocations, e.g., different countries. In one embodiment, thecommunications distribution system 800 may use packet-based networks,e.g., Ethernet, for signaling purposes only. In other words, separatevideo networking, e.g., the video bus 810, may be provided to facilitatehigh data transfer rates for streaming video and audio.

[0122] The TELCO network 850 may be used for non-packed basedcommunications, e.g., standard telephone voice-only communications, aswell as (relatively) lower quality telephone based video signals, e.g.,where a video frame is transmitted once every several seconds, such asby a video cell-phone. This kind of video communication may be used inthe event that IP-based video service is not available. As is wellknown, the TELCO network 850 may also provide means for telephonepacked-based long distance communications, e.g., standard digitaltelephone communications, such as ISDN or SS7.

[0123]FIG. 8B—A Content Distribution System

[0124]FIG. 8B is a high level diagram of a content distribution system,according to one embodiment. As used herein, the term “content” refersto any type of media content, especially image content (still and/ormoving), e.g., video content, including, for example, television contentfrom regular television broadcasts, cable, and satellite sources, analogvideo signals (possibly including accompanying audio signals), such asNTSC, PAL, SECAM, RGB, Y/C (e.g., S-Video), YUV, etc., and digital video(and possibly audio) sources and formats, such as DVD, VTR,computer-generated signals, MPEG (2, 3, 4, etc.), \CCIR 601, D1, D2, D4,and 4:2:2, among others. In other words, essentially any video sourceand format may be acceptable as a content source. Various aspects of thecontent distribution system are substantially the same as in thecommunication distribution system described above with reference to FIG.8A, thus, descriptions of components already described above may beabbreviated.

[0125] As FIG. 8B shows, similar to the system of FIG. 8A, the contentdistribution system may include co-located computer system 802, e.g.,multiple blade computers 102, coupled to the plurality of humaninterfaces 130 through video manager 800B. As described above, the videomanager 800B may include blade switch 809 which provides switchingfunctionality for the blade computers 102, and video switch 808. Alsosimilar to the system of FIG. 8A, in one embodiment, the co-locatedcomputer system 802 may be coupled to cluster computer system 803, asdescribed above. The co-located computer system 802 may also couple tonetwork switches 812 which may provide access by the co-located computersystem 802 to networks, e.g., LAN, WAN, the Internet, etc., as shown,and described above with reference to FIG. 8B.

[0126] As shown, in one embodiment, the video switch 808 may couple to acontent processor 806, described in more detail below. The contentprocessor 806 may receive input from a plurality of content sources 860,such as television tuners, satellite tuners, cable tuners, digital videosources, etc., as mentioned above. In one embodiment, in addition tocoupling to the co-located computer system 802 via the video and bladeswitches (808 and 809), the content processor 806 may also coupledirectly to the co-located computer system 802 via a transmissionmedium, e.g., a cable 811. This cable 811 may be used as a command lineto allow users or blade computers to control the content processor 806,e.g., to choose content for display at the various human interfaces 130.

[0127] In one embodiment, the content distribution system may alsoinclude the communication distribution system described above withreference to FIG. 8A. In other words, the content distribution systemmay operate in conjunction with (or integrated with) the communicationsdistribution system described above, where media content from a varietyof sources may be presented by human interfaces in tandem withcommunication signals (e.g., audio and/or video) from other users of thesystem (and/or external systems). Such a system is described below withreference to FIG. 8C. Further details of content distribution areprovided below with reference to FIGS. 9-24.

[0128]FIG. 8C—A Content and Communications Distribution System

[0129]FIG. 8C is a high level diagram of a content and communicationsdistribution system, according to one embodiment. The content andcommunications distribution system may combine the functionality of thecommunication distribution system of FIG. 8A with the functionality ofthe content distribution system of FIG. 8B, integrating the two systems(and sharing major components) to provide The content and communicationprocessors and their operations are described in more detail below.

[0130] As FIG. 8C shows, in one embodiment, the content andcommunications distribution system 800 may include co-located computersystem 802, which preferably includes a plurality of blade computers102, coupled to human interfaces 130 through blade switch 809 and videoswitch 808. In the embodiment shown, the computers (blades) 102 areincluded in a plurality of rack-mounted cages 511, although otherembodiments are also contemplated. In one embodiment, the blade switch809 and the video switch 808 may be included in a backpack or functionmodule that couples to the blade cages 511, as described in detail inU.S. patent application Ser. No. 09/728,669 titled “A System ofCo-Located Computers in a Framework Including Removable Function Modulesfor Adding Modular Functionality” which was incorporated by referenceabove. It is noted that the video switch 808 preferably includes aplurality of blade video switches. In other words, in one embodiment,the video switch 808 may include respective video switches for each ofthe plurality of blades and/or human interfaces.

[0131] As described above, in one embodiment, the co-located computersystem 802 may be coupled to a second co-located computer system 803,e.g., a cluster computer system 803, for burst processing and/orreplacement blade services, as desired. The co-located computer system802 may also couple to network switches 812 which may provide access bythe co-located computer system 802 to networks, e.g., LAN, WAN, theInternet, etc., as described above.

[0132] As mentioned above, each human interface 130 may be at a remotelocation from the co-located computer systems 802, and may include anyof various human interface devices, including any of one or moremonitors 116, a keyboard 112, a mouse 114, joystick, trackball, audiospeakers (or headphones), a microphone, a printer, a scanner, atelephone, a removable storage medium, a biometric sensor, a barcodereader, a VR (Virtual Reality) interface device, and a PDA (PersonalDigital Assistant) IR (Infra-Red) device, among others. As FIG. 8C alsoshows, in a preferred embodiment, one or more of the human interfacesmay include a camera 135 which may be used for video conferencing,telepresence applications, videophones, etc.

[0133] As shown, in one embodiment, the video switch 808 may couple toboth the communications processor 804 and the content processor 806,both of which are described in more detail above with reference to FIGS.8A and 8B, respectively. As mentioned above, the content processor 806may receive input from a plurality of content sources 860, such astelevision tuners, satellite tuners, cable tuners, digital videosources, etc., and may insert the video signals corresponding to thecontent into images transmitted to respective monitors. For example, astreaming news broadcast may be inserted into the screen image of amonitor (in a human interface) as a Picture-in-Picture (PIP) image,allowing the user of the human interface to view (and possibly hear)relevant media content while working.

[0134] As also mentioned above, the communication processor 806 maysimilarly insert communications images and/or sounds (and/or text) intosignals transmitted to the human interface 130, including telephonesignals, text messages, video images, such as from a camera 135, or anyother type of communications signal. In one exemplary use of the system,a plurality of users may each send and receive camera images (andaccompanying audio) of each other in a video conferencing session. Theserespective video images may be inserted into each of the otherparticipant's screen images, allowing the participants to view (andoptionally hear) one another via their respective human interfaces.Additionally, media content, such as television broadcasts, may also beinserted into the screen images, allowing each participant to viewrelated video content during the video conference.

[0135] In another exemplary application of this embodiment of thepresent invention, a brokerage house may provide each analyst with ahuman interface, as described above. Each analyst may participate in avideo conference session with other analysts, where the analyst may viewand/or communicate with the other participants via the communicationprocessor. Simultaneously, each analyst may also view one or morereal-time video streams, such as, for example, business news broadcasts(e.g., from CNN, Headline News, MSNBC, etc.), streaming stock quotes,e.g., from the Internet or a dedicated subscription service, an soforth, provided by the content processor. In addition to theseinformation sources, each analyst may also (simultaneously) use one ormore software programs running on one or more of the networked bladecomputers 102 to perform related analyses. Thus, each analyst may haveaccess to a wide variety of information sources and channels whereby heor she may more effectively make decisions, perform research, and/orcommunicate with others in an integrated fashion.

[0136] In a slightly different approach, a user may utilize the multipleinformation sources and channels to multi-task. In other words, thevarious content and communication sources accessed by the user may notbe related to one another, but instead may relate to two or more tasks,issues, or applications. For example, a manager may maintain a pluralityof information streams from respective divisions or departments of anenterprise to keep abreast of many ongoing operations simultaneously.

[0137] In yet another application of the system of FIG. 8C, the varioususer interfaces may be configured hierarchically to reflect thedifferent levels and positions in an enterprise. For example, in amanufacturing operation, one or more floor managers may receiveinformation streams from the manufacturing process itself, e.g., viamonitors, and/or from supervisors or operators on the plant floor. Eachfloor manager may also be in communication with his or her immediatesuperior, e.g., a department manager.

[0138] Similarly, each department manager may receive communicationsand/or content from one or more floor managers, and may also participatein a video conference session (or other type of session) with the otherdepartment managers. Additionally, each department manager may be incommunication with his or her superior, e.g., a division manager. Thus,each layer of management may send and receive information from the layerabove and the layer below (and other layers as needed). In this manner,information flow in the enterprise may be configured, integrated, andmanaged as desired to facilitate efficient communications anddecision-making. More specifically, hierarchical use of the describedsystems and methods may provide powerful means for seamlessly andorganically integrating information at many levels and resolutions inthe enterprise. For example, at each respective level, information maybe assessed, analyzed, and integrated to generate new information whichmay then be transmitted to the other levels, e.g., the level directlyabove the respective level. Conversely, strategic and tacticalmanagement information, e.g., directives and/or goals, may propagatedownward through the levels, where at level the received directives maybe translated into more specific or lower level directives which maythen be transmitted to subordinate levels.

[0139] Thus, various embodiments of the systems described herein mayprovide an infrastructure for pervasive integrated information flow inan enterprise, resulting in improved operations and decision-making.

[0140]FIG. 9—Block Diagram of a Content and Communications DistributionSystem

[0141]FIG. 9 is a block diagram of one embodiment of a content andcommunications distribution system. More specifically FIG. 9 provides ahigh-level illustration of the system as it relates to operationsregarding single blade computers 102A and 102B and respective humaninterfaces 130A and 130B. Although the system diagram of FIG. 9corresponds to the system described above with reference to FIG. 8C, itshould be noted that respective portions of the diagram and descriptionalso apply to the systems of FIGS. 8A and 8B.

[0142] As FIG. 9 shows, in this embodiment, the distribution system mayinclude three primary components, namely, the communications processor804, the content processor 806, and the video switch 808, used by bothprocessors. These components may be networked together through an analogmulti-channel video bus (VBus) system 810. The communications processor804 and the content processor 806 may coupled to each blade computer 102and the corresponding human interface 130 through the video switch 808as shown. It is noted that this VBus system 810 may stream video (andpossibly audio) content based on production video technology rather thanpacket-based computer technology, due to the substantially higherbandwidths provided by the former. In other words, as mentioned above,delivery of content and communications imagery to the user's display(e.g., monitor(s)) may be provided by a PIP insertion process (which maybe analog PIP insertion, digital PIP insertion, or both), as opposed torelying on the blade computer's video system to generate the images, thecomputer's packet-based networking system (e.g., Ethernet) to transferthe images, and the computer's CPU to process the images. This divisionof functionality may substantially increase the effective bandwidth ofthe system.

[0143] As FIG. 9 also shows, the video switch 808 may couple to eachhuman interface 130 through blade switch 809 which may couple to devicesin each HI 130 via a transmission medium such as Category 5 cable oroptical fiber, among others. In one embodiment, the blade switch 809 maycouple to the human interface devices (e.g., monitor, mouse, keyboard,etc.) through an extender device 121B as described above, allowingremote placement of the human interface 130. For more details regardingextension of functional distances for remote human interfaces, pleasesee U.S. Pat. No. 5,764,924 titled “Method And Apparatus For Extending ALocal PCI Bus To A Remote I/O Backplane”, whose inventor is Soon ChulHong, which is hereby incorporated by reference, and U.S. patentapplication Ser. No. 09/619,989 titled “System And Method For ProvidingA Remote Universal Serial Bus”, whose inventors are Dan Barrett, MikeBarron, and Andrew Heller, which was incorporated by reference above.

[0144] As mentioned above, in one embodiment, the video switch 808 mayinclude a plurality of blade video switches corresponding to theplurality of blade computers 102. Each blade video switch may operate ona “per blade computer” basis, providing connectivity to one or more ofthe video buses 810, as shown. For example, in one embodiment, each cage511 may include 8 blades, and may couple to or include (e.g., in theform of a backpack or function module) a video switch 808 comprising 8blade video switches, one per blade 102 in the cage 511. Similarly, inone embodiment, each cage 511 may couple to or include (perhaps in thesame function module) an 8×8 blade switch 809 which may be operable toprovide for switching between any of the 8 blades in the cage 511.

[0145] In one embodiment, the video buses (VBuses) 810 may include avideo network for the co-located computer systems 802, where each VBusmay include a three-wire bus common to one or more of the blade videoswitches. Each VBus may be operable to carry Y/C video signals andbi-directional command signals, where the Y/C video signals (alsoreferred to as S-Video) include luminance (Y) and chrominance (C). Inother words, the three wires may be used to transmit the Y, C, andcommand signals, respectively. Additionally, in one embodiment, thecommand line (wire) may also be used to transmit audio signalsassociated with the video images. For example, the command signals maybe transmitted on a 100 KHz carrier, while the audio signals may betransmitted on a base band. The audio signal may thus automaticallyaccompany the corresponding video signal.

[0146] Each VBus may be selectable by a blade video switch correspondingto the blade computer 102, and may then provide video and audio (and/orother) signals to that blade computer 102. Image signals on the selectedVBus (e.g., on the command line of the VBus) may be accompanied byinstructions specifying the size and location of the image to berendered on the user's screen. For example, the instructions may includea Start X, Start Y, Offset X, and Offset Y indicating the placement ofthe image on the monitor screen of the user's human interface. As FIG. 9shows, the system may also include a common communication line, referredto as a primary control bus 920, common to all of the blade videoswitches, which may be used to transmit the VBus assignments for eachvideo switch 808. Thus, the master control bus 920 may transmit signalsto the blade video switch indicating which VBus to connect to, whilespecific information regarding the image size and location may beprovided by the command line of the specified VBus.

[0147]FIG. 10—Signal Cabling for the Co-located Computer System

[0148]FIG. 10 illustrates one embodiment of signal cabling for theco-located computer system 802. In the embodiment shown, the co-locatedcomputer system 802 includes a plurality of cages, e.g., installed inone or more racks 1000, where each cage may include a plurality of bladecomputers 102. As FIG. 10 shows, communications and content signals maybe transmitted to and from each of these blade computers 102 via aribbon cable 1010 with mass termination plugs, where each masstermination plug couples to a respective cage. The ribbon cable 1010 maythus include the video bus (VBuses) 810 of FIG. 9, providing forcommunication between the co-located computer system 802 and thecommunications and content processors 804 and 806, as shown.

[0149] As FIG. 10 also shows, the ribbon cable 1010 may also couple tocages 511 in additional racks, thereby providing signal connectivity tothe blade computers 102 installed therein. For example, referring backto FIGS. 8A-8C, the ribbon cable may couple to the cages/blades of theco-located computer system 802, and may also couple to the clustercomputer system 803, e.g., to the cages/blades of the cluster computersystem 803. As indicated by FIG. 10, in the embodiment shown, the ribboncable may include multiple signal paths and ID mass terminations runningdown the back or side of the rack 1000, attaching to each of the cages511. Each cage 511 may then provide connectivity to the individual bladecomputers 102 in the cage 511. It should be noted that the signalcabling shown is intended to be exemplary, and is not intended to limitthe particular signal cabling for the co-located computer system 802 toany particular embodiment.

[0150]FIG. 11—Video Switch

[0151]FIG. 11 is a block diagram illustrating one embodiment of thevideo switch 808. As described above, the video switch 808 may provideswitching functionality between a respective blade computer 102 and eachof the plurality of VBuses 810. In other words, for each blade computer102, the video switch 808 may select a VBus 810 as a video source ofcontent and/or communication signals for that blade computer 102.Additionally, control functions provided by the video switch 808 mayinclude capturing and storing one or more video images in memory, e.g.,received from the selected VBus 810; changing the format, protocol,size, and/or resolution of the received video images as needed fordisplay on a monitor of the target human interface; and insert thestored image or sequence of images as PIP events into the monitor'ssignal stream.

[0152] As FIG. 11 shows, a control module 1104 may couple to the mastercontrol bus 920, and may be operable to receive VBus selection signalsfrom the master control bus 920 indicating from which VBus 810 toreceive video content or communications. Note that although only two(three-line) VBuses, 810A and 810N, are shown, the system generally willinclude more than two VBuses. Each of the three lines of a respectiveVBus 810 may couple to a respective VBus line switch 1111. For example,in the embodiment shown, the Y line of each VBus 810 couples to VBusline switch 1111A, the C line of each VBus 810 couples to VBus lineswitch 1111B, and the command line of each VBus couples to VBus lineswitch 1111C. As FIG. 11 indicates, the control module 1104 may coupleto each of the VBus line switches 1111, and may be operable to provideVBus selection signals to the VBus line switches 1111 indicating theselected VBus 810.

[0153] Once the VBus 810 has been selected, signals may be transmittedfrom the communications processor 804 and/or the content processor 806over the selected VBus 810. The VBus line switches may pass the receivedvideo signals (e.g., the Y and C signals) through respective A/Dconverters 1101A and 1101B, as shown, thereby converting the analogvideo signals into digital signals. In the embodiment shown, the Y and Canalog signals are converted into respective 6-bit signals and combinedinto a single 12-bit signal. In other words, the Y and C digital signalsmay be combined such that a 12-bit number represents each pixel in thevideo image. The received command signal on the selected VBus 810 may bepassed on by VBus line switch 1111C to the control module 1104, asshown.

[0154] The 12-bit pixel values may be stored in memory 1106 for laterplayback. In one embodiment, storing the pixel values (e.g., the image)in memory may include sizing, scaling, and/or cropping the image basedon the command signals received from the command line of the source VBus810. These image processing functions may be performed simply by mappingthe pixel values into relative locations in the memory 1106. In otherwords, the control module 1104 may perform the various image processingfunctions by placing received pixel data into relative memory locationssuch that the stored image is of the desired size, protocol, and/orresolution.

[0155] As FIG. 11 also shows, video signals from a blade computer 102may be received in the form of RGB (Red/Green/Blue) signals, representedby Blade-R, Blade-G, and Blade-B in the bottom left of the Figure. Thesesignals are targeted for display on the monitor(s) of the target humaninterface 130. At playback, the controller module 1104 may be operableto retrieve the stored 12-bit words representing the stored image pixelsin Y/C format, and convert the 12-bit YC value (S-Video) to a 24-bit RGBvalue. In one embodiment, this conversion may be performed via a RAMlook-up table 1105, where the 12-bit YC value is placed on the addressbus of the RAM 1105, and the corresponding 24-bit (3×8) RGB valuereturned. In other words, the RAM table 1105 may be configured such thateach Y/C value corresponds to an address in the RAM 1105, and when theY/C value is fed to the RAM address bus as an address, the RAM 1105returns the corresponding 24-bit RGB value.

[0156] As shown, the three 8-bit RGB signals may then be converted toanalog via respective D/A converters 1103. The control module 1104 maythen insert the RGB signals into the blade computer's RGB analog videostream via RGB line switches 1112 as specified by the command signals.In other words, the image insertion may be performed in accordance withscreen position instructions included in the received command signalsmentioned above, generating combined or modified analog RGB videosignals which may then be transmitted to the video encoder 123 of thehost extender device 121A, which in this embodiment, includes respectiveencoder components for the R, G, and B video signals, as shown. Furtherdetails of the video signal insertion are provided below with referenceto FIG. 12.

[0157] In one embodiment, the video signals received over the selectedVBus 810 may originate from either the communications processor 804 orthe content processor 806, but not both, with the video source (eitherthe communications processor 804 or the content processor 806) specifiedand selected, for example, by the control module. However, in otherembodiments, video signals from both processors 804 and 806 may bemultiplexed, e.g., by the video switch 808, such that both content andcommunications information may be streamed to the monitor screen at thesame time. However, it should be noted that doing so may result in lowerframe rates for the image streams. In one embodiment, multiple sets ofVBus line switches 1111 (and corresponding A/D converters 1101) may beused to handle the increased video stream load. Multiplexing content andcommunication video streams may, for example, facilitate a user watchinga news broadcast and teleconferencing about the broadcastsimultaneously.

[0158] As noted above, the video switch 808 may service both the contentand communications processors. When servicing the content processor 806,a sequence of video frames may be inserted into the screen image, asdescribed above. However, when servicing the communications processor804, where multiple images from respective cameras are streamed, e.g.,in a video conferencing session with three or more participants, thestreaming video may include a plurality of sub-frame images packedtogether as a single frame. The video switch 808 may deconstruct thecommunications frame into the constituent smaller images. Furtherdetails of this process are described below.

[0159]FIG. 12—Image Scan-line Insertion

[0160] As described herein, the video manager may operate to selectivelyinsert video data (analog or digital) from an external source into thevideo data (analog or digital) output from the computing system fordisplay, e.g., using a picture-in-picture mechanism. Thepicture-in-picture mechanism may operate on digital video data, analogvideo signals, or both.

[0161] In one embodiment, a first computing system may be operable togenerate first video data for display on a display device in a firsthuman interface, where the first video data corresponds to a firstimage, and where the first video data is arranged in a scan line format.The video manager may be operable to receive second video datacorresponding to a second image from a source, and insert the secondvideo data into a location in the first video data.

[0162] In a first embodiment, the first video data and second video datacomprise digital data. As one example, the video manager may operate toselectively access portions of the first video data and second videodata from different portions of memory, e.g., on a scan line basis, andprovide the combined digital data for display. The digital video datamay be combined in various ways. One method for combining the digitalvideo data is described in U.S. Pat. No. 6,067,098, incorporated byreference above. The combined digital data may be transmitted fordisplay, or alternatively the combined digital data may be provided toD/A converters for conversion to analog video signals for display.

[0163] In a second embodiment, where the second video data comprisesanalog video signals, the second analog video signals are inserted “onthe fly” into the first analog video signals corresponding to the firstvideo data as the first analog video signals are output from thecomputing system. The combined first and second analog video signals(referred to as third analog video signals) are transmitted across thecommunication medium to the remote human interface. The display deviceof the first human interface may then operate to display a third imagebased on the third analog video signals.

[0164] The following provides greater detail regarding the secondembodiment discussed above. However, it is noted that embodiments of thepresent invention may operate to combine digital or analog video signalsto achieve the benefits described herein.

[0165]FIG. 12 illustrates image scan-line insertion, according to oneembodiment. More specifically, FIG. 12 illustrates the insertion of RGBimage lines 1204 from memory into RGB image lines 1202 from a bladecomputer 102 via switching, as described above with reference to FIG.11. As FIG. 12 shows, in accordance with position instructions includedwith the original video signals (from the selected VBus 810), the RGBline switches 1112 (see FIG. 11) may time their respective switchingsuch that the RGB lines (or portions of the lines) from the bladecomputer may be replaced at the correct position on the monitor screenwith the RGB image lines 1204 from memory. As shown in FIG. 12, thisimage insertion may result in a Picture-In-Picture screen image.

[0166] Timing

[0167] In managing multiple video streams from content andcommunications sources, several timing issues may arise. For example,video streams from content sources are likely to be asynchronous withrespect to one another, as well as to the video processing and displayin the distribution system. Additionally, content video signals fromcontent sources, such as television (TV) tuners, are generallytransmitted in interleave raster scan format, while many computermonitors display images using a progressive raster scan. These issuesare addressed below.

[0168]FIG. 13—Frame Timing

[0169]FIG. 13 illustrates video frame timing issues related to thetransmission and display of content and communications video streams.One frame timing issue relates to the difference between the frame rateof the video stream from the VBus 810, i.e., the delivery frame rate andthe refresh rate of the monitor on which the video stream is displayed,i.e., the display frame rate. Another frame timing issue relates to thedelivery and display of video streams from multiple VBuses 810. In thedescriptions which follow, it is assumed that communications images arerendered on the user's monitor screen in a form similar to contentimages.

[0170] As FIG. 13 shows, parallel asynchronous video frame sequences maybe transmitted at roughly 30 frames per second (fps) on respectiveVBuses 810A-810C. This asynchronicity is represented by the three starttimes of each frame sequence, Time 1, Time 2, and Time 3. Also shown arerespective display frame sequences for a single monitor read 1302 at 75fps, a dual monitor read 1304 at 37.5 fps, and a quad monitor read 1306at 18.75 fps.

[0171] In the example illustrated in FIG. 13, the refresh rate of asingle monitor is assumed to be 75 fps, as mentioned above. Thus, theframe read speed may be considered to be substantially constant. Inother words, the time necessary to read a frame for presentation to ascreen may be the same regardless of the number of monitors in the humaninterface 130. However, the frequency of reads for display on a givenmonitor may depend upon the number of monitors in the human interface130. For example, images displayed on only one of four monitors may beread from memory at only ¼ the frequency of the single monitor feed, asillustrated by the quad monitor read 1306 in FIG. 13.

[0172] It should be noted that although the frame rates of thetransmitted video frame sequences are shown as 30 fps signals, ingeneral, these frames are not provided at exactly 30 fps, i.e., eachvideo stream generally has a slightly different frame rate, typicallywithin 0.001% of 30 fps. As a consequence, roughly every 5 minutes aframe in one stream may creep by a frame in another stream. It is alsonoted that in actual operation, the relative time differences betweenthe different video streams may constantly vary.

[0173] In one embodiment, the differing frame rates may be handled bybank-switching the frame read and write processes. The frame write(input) A/D conversion and memory management process may operateindependently of the frame read (output) D/A conversion and memorymanagement process. Each process may notify the other as to when thebank shift can occur, thereby gating each process. In this approach, theuser's monitor may at times display the same content image more than onetime or miss a content frame. These effects may not be noticeable to theuser due to the size of the displayed images, the presence of othercontent or communications images, etc.

[0174]FIG. 14—Interleave and Progressive Scan Lines

[0175] In general, TV and TV-like content images are transmitted anddisplayed in an interleaved manner. In contrast, most modem computermonitors display images in a progressive scan. FIG. 14 illustrates bothof these scanning approaches in highly simplified 6 scan-line examples.The scan lines are numbered in the order in which they are drawn.

[0176] As FIG. 14 shows, in an interleaved scan 1402, every other lineof a frame is drawn to the screen, e.g., the odd lines, until the end ofthe screen is reached, at which point the raster scan resets to the topof the screen and draws the remaining lines, e.g., the even lines. Thus,the second line viewed, i.e., the second screen line, is actually thefourth scan line. As FIG. 14 also shows, in a progressive scan 1404, theentire frame is drawn line by line from start to finish. Thus, comparingthe two processes in parallel, where a single video stream is displayedby both methods, the image data drawn by the fourth scan line (thesecond line from the top) in the interleave case 1402 may be almost, butnot exactly, the same as the image data drawn in the second scan line(the second line from the top) of the progressive process 1404. Thereason the scan line data from the two corresponding screen lines maydiffer is that every other screen line of the interleaved image is drawnwith a half-frame time delay compared to the corresponding screen linesof the progressive scan. If the source image data change during thattime, then the screen lines corresponding to the “second pass” scanlines will reflect the new data, and so may differ from correspondingscreen lines in the progressive scan. In other words, a progressive scanmay be compared to a movie frame, in that the frame is revealed to theeye in a vertical progression, like a shutter moving in a movieprojector, while the interleave scan may actually include twohalf-images with a slight respective temporal delay. Thus, the fact thatthe content video stream may be received in interleaved format, butdisplayed in progressive format could be problematic.

[0177] In one embodiment, this issue may be resolved by writing theinterleaved image to memory such that the successive scan lines arestored or mapped as sequential lines to be read as a progressive scan.In other words, the interleaved image may be written to memory as if itwere being written to screen, after which it may be read with aprogressive scan and displayed with a progressive scan. Said anotherway, the interleaved image may be buffered in the format of a screenimage, and subsequently read and displayed in a progressive fashion.FIG. 14 may illustrate this process by interpreting the interleavepicture 1402 as a write to memory operation, and the progressive picture1404 as a subsequent read from memory operation.

[0178]FIG. 15—Image Resolution Reduction

[0179] In many cases, images received from the VBuses 810 at oneresolution may be displayed at a different, generally lower, resolution.This “de-res” may be performed in more than one way. For example, in oneembodiment, the received pixel data of an image may be sampled forwrites to memory, e.g., only storing every third pixel (for a ⅓ imageresolution reduction). In other words, each sample pixel is assumed torepresent that portion of the image in its vicinity. However, there maybe substantial, i.e., noticeable, image degradation as a result. Inanother embodiment, e.g., where sufficient computation resources areavailable, each sampled pixel may be modified by averaging pixels in theneighborhood, e.g., by averaging the pixel value with those of pixelsabove, below, left, and right of the sample pixel. FIG. 15 illustratesboth of these approaches.

[0180] As indicated by FIG. 15, in the example single pixel de-resprocess shown, every third pixel in a scan line is selected, e.g., forstorage in memory and subsequent display on a monitor screen. Similarly,the pixels are sampled from every third image line. In the case of acorresponding averaged pixel de-res process, every third pixel (fromevery third image line) is selected and averaged with the pixels in aneighborhood of a specified size, in this case, a 3×3 pixelneighborhood. This technique, well-known in the art, may provide areduced image of substantially higher quality than the single pixelde-res process, in that the modified or averaged pixel value does infact represent the portion of the image in its vicinity.

[0181] FIGS. 16A-16E—Image Scaling

[0182] Monitors included in the human interfaces of the communicationsand/or content distribution system may have any of a variety of screensizes, such as, for example, 1024×768, 1280×1024, and 1600×1200, amongothers. For each of these screen sizes there may be optimal sizes forthe inserted communications and/or content images. In other words, theremay be certain scaling factors which when applied to the images resultin image sizes which may be particularly suitable for display on theuser's screen(s).

[0183] FIGS. 16A-16E illustrate a number of example image insertioncases for a 1024×768 monitor screen. FIG. 16A illustrates a singlefull-sized VGA or TV screen image (640×480) inserted into the screenimage, while FIGS. 16B-16E illustrate various arrangements of multipleinserted images on the screen, where the size of each inserted image isdisplayed along with the fraction of a full-sized VGA/TV screen imagethat the inserted image occupies, i.e., the relative size of theinserted image to a full-sized VGA/TV screen image. In one embodiment,the fraction associated with each image may also be interpreted as theeffective scaling factor need to reduce a full VGA image to thecorrectly sized insertion image.

[0184] For example, FIG. 16B illustrates two inserted images of size160×240, where each inserted image is ⅛ the size of a VGA screen. Thus,assuming that the original size of the insertion image was 640×480, aneffective scaling factor of ⅛ has been applied to the image to achievethe proper reduction in inserted image size. The reason the scalingfactor is termed “effective” is that depending upon the scaled insertionimage aspect ratio (horizontal to vertical ratio or its inverse) thesize reduction of the image may be achieved in different ways. Forexample, the insertion image of FIGS. 16B, 16C, and 16E are each in“portrait” orientation, i.e., are taller than they are wide, and thus,depending on the orientation of the original images, the image reductionmay involve cropping as well as scaling the images. In contrast, thereduced images shown in FIG. 16D are each in “landscape” format with areduction ratio of ¼, and so a simple scaling/re-sizing operationperformed on the original images may suffice.

[0185]FIG. 17—Content Processor

[0186]FIG. 17 is a block diagram of one embodiment of the contentprocessor 806. In one embodiment, the functions of the content processormay include converting the various formats and protocols of contentvideo stream signals to simple S-Video (Y/C) format, and to assign theresulting Y/C signal to a particular VBus 810.

[0187] As FIG. 17 shows, in this embodiment, one or more media inputs1720 may be coupled via switch 1701A to analog protocol converters 1704and audio processor 1712. The media inputs 1720 may include one or moremedia sources, such as, for example, modulated signal sources such as aregular broadcast TV tuner, a cable tuner, and/or a satellite tuner,among others. Audio signals associated with the video image signals maybe sent separately (from the video image signals) to the audio processor1712 as shown. The audio processor 1720 may process the audio signals,e.g., by converting to and/or from any of various industry standardaudio protocols such as, for example, Dolby 5.1, Composite Stereo (FMStereo), Midi Commands, DS (Digital Stereo) 1, 2, and 3 protocols fordigital sound, among others, and may then transmit the audio signals toVBus switch 1702, where command signals received from controller 1708may determine onto which VBus 810 the signals may be loaded ortransmitted. As indicated by FIG. 17, the command signals from thecontroller 1708 may be sent to the switch 1702 in response to commandsreceived by the controller 1708 from the master control bus 920.

[0188] Other content signals received by the content processor 806 mayinclude both analog inputs 1730, including composite signals such asNTSC, PAL, SECAM, and component signals such as RGB, Y/C (e.g.,S-Video), YUV, etc., as well as digital video (and possibly audio)sources and formats 1740, such as DVD, VTR, computer-generated signals,MPEG (2, 3, 4, etc.), \CCIR 601, D1, D2, D4, and 4:2:2, among others, asmentioned above with reference to FIG. 8B. As shown, these analog anddigital sources may be switchable by respective content switches 1701Band 1701C, where the selected analog inputs may be routed to the analogprotocol converters 1704, and the selected digital inputs may be routedto digital protocol converters 1706. The digital protocol converter 1706may transmit the converted digital signal to a D/A converter 1103D,where the digital signals may be converted to analog signals. Whilesupport for such a wide variety of protocols and formats may seemdifficult to achieve, it is noted that they are generally industrystandards, and a wide variety of low-cost chip-level converters andprocessors are readily available from major chip manufacturers toperform these conversion tasks. The respective converters 1704 and 1706may convert the received signals from their respective originalformats/protocols to a simple Y/C format and transmit the convertedsignals to Y/C unit 1707 where the switch 1702 may then place the videosignals onto a selected VBus 810, as shown.

[0189] Thus, as FIG. 17 shows, the switch 1702 may load the Y/C videosignals and the corresponding audio signals onto the selected VBus 810for delivery to the video switch 808, as described above. As alsodescribed above, each VBus 810 may include three lines—two for the Y/Cvideo signals and a command line for command instructions and audiosignals. Thus, in one embodiment, the command line may server twopurposes. The command line may carry signaling information (the commandsignals) which may be modulated onto a 100 KHz carrier, and may alsocarry the audio signals associated with the video signals as a base-bandsignal. In one embodiment, the audio signal may be transmitted viacurrent modulation rather than voltage modulation to facilitateaggregation of audio signals. This approach may be particularly usefulin using the VBus 810 for audio in the communication mode, i.e., whenthe distribution system is used to distribute communication signalsbetween users.

[0190] FIGS. 18-22—Communications Processor

[0191] FIGS. 18-22 illustrates various aspects of the communicationsprocessor 804 and its operations, according to one embodiment. Asmentioned above, the communications processor 804 preferably operates inconjunction with the content processor 806 in a communications andcontent distribution system, although a stand-alone communicationsdistribution system is also contemplated. The communications processor804 may be considered part organizer and part scheduler, in that thecommunications processor 804 may organize sub-frames from various usersfor display on the monitors of the human interfaces, as well as scheduleframe events and/or multi-user conference sessions.

[0192] As noted above, the distribution system Video Bus 810, alsoreferred to as a collective video bus, preferably includes a pluralityof video buses (VBuses), each including three lines or wires. In oneembodiment, the collective video bus 810 may be 32 channels wide, i.e.,may include 32 VBuses, and so may include roughly 100 lines (e.g., 96lines, 32×3). As described above with reference to FIG. 10, in oneembodiment, the collective video bus 810 may include a ribbon cablerunning the height of each rack, connecting each cage 511 in parallel,thereby providing full access for each cage 511 to each of the VBuses.

[0193] As also noted above, for bandwidth reasons, video streaming maybe accomplished with analog video signals encoding each video stream asa sequence of image frames. In other words, packet-based video streamingwould require substantially greater bandwidth. Also, using thepacket-based data network (e.g., Ethernet 840B and 807) for videostreaming would likely consume the data network's bandwidth, degradingboth the (packet-based) video signals and packet-based data signals usedby the communications processor.

[0194]FIG. 18—Communications Frames

[0195] In one embodiment, the communications processor 804 may handlevideo streaming in a similar manner as the content processor. However,in the communications system, each frame may include a number ofindependent smaller sub-frames. FIG. 18 illustrates one embodiment of asequence of communications video frames, where each frame includes fivesub-frames. It should be noted that the frames shown are meant to beexemplary only, and are not intended to limit the format or the numberof sub-frames in a frame to any particular format or number.

[0196] In the example of FIG. 18, each of the five sub-frames is assumedto be an image of a user in a multi-user communication session or videoconference. As FIG. 18 shows, in this embodiment, in addition to thefive sub-frames, each of the frames also includes a small “dead time”interval, labeled “RS” for “Re-Sync” time, described below. When thesession is initiated, or when a participant is dropped or added to thesession, referred to as a communications event, the number of sub-framesin each frame may increase or decrease accordingly. In one embodiment,the sub-frames may occupy time slots in a time domain multiplexingscheme, similar to the approach used in regular telephone conferencecalls.

[0197] Thus, each user participating in the session may be allocated asub-frame. Because the total number of pixels in the frame cannot exceedthe number of pixels on the user's screen, the individual sub-framesmust be smaller than the frame. Thus, in one embodiment, the number ofpixels in each sub-frame may be specified at the beginning of thesession. For example, this information may be passed to the respectivecamera 135 generating the sub-frame image such that the image generatedby the camera 135 is sized by the camera 135, i.e., the image size maybe set at the origin (the camera 135) as opposed to sending thefull-sized camera image and de-resing the image when it is received.This approach may thus eliminate the need for constant re-sizingcomputations by the system, offloading this work onto the respectivecameras 135.

[0198] In a communications session, when the time slot for a sub-framefor a specific user's image occurs, the video bus 810 may switch to ahigh-impedance (Hi-Z) mode, and the user's image (from the camera 135)may be added to the frame as a sub-frame. Thus, the frame may begenerated “on-the-fly” during the communications session, as opposed tosending all of the participant images to a controller, combining theminto the frame, then transmitting the finished frame.

[0199] FIGS. 19-21—Transmitting User Interface Signals to and from theBlade Computer

[0200] FIGS. 19-21 illustrate an approach whereby video and audioinformation may be transmitted, i.e., returned, from a user's desktopover the same cable used to deliver video and HID (Human InterfaceDevice) signals to the user's desktop. This approach may eliminate theneed for a second cable to accomplish the return signal task.

[0201]FIG. 19—Spectrum of User Interface Signals to and from the BladeComputer

[0202]FIG. 19 illustrates spectrum allocation for user interfacesignals, according to one embodiment. In one embodiment, a USB signalreturn line may be employed using distance extension technology (e.g.,USBX) over Category 5, 6, or 7 Communications Cable to simultaneouslycarry the audio and video information back to the computer. In thisapproach, four pairs of wire may be used to carry signals in BalancedMode. One pair may be used for each color (Red, Green, and Blue), andthe final fourth pair may be used for bi-directional USB signaling. Forfurther details regarding extended USB signaling, please see U.S. patentapplication Ser. No. 09/619,989 titled “System And Method For ProvidingA Remote Universal Serial Bus”, which was incorporated by referenceabove.

[0203] As FIG. 19 shows, the spectrum for user interface signalstransmitted to the blade computer 102 may be divided thus: video signalsmay be transmitted in the frequency range of approximately 100 KHzthrough approximately 2 MHz; microphone signals may be transmitted inthe frequency range of approximately 4-6 MHz; and USB signals may betransmitted in frequencies from approximately 6 MHz upwards.Additionally, in one embodiment, control information may be embedded inthe downstream audio signals to configure the camera at the desktop.Thus, in one embodiment, the USB signals from the remote extender device121B, e.g., the blade encoder, may be partitioned into 1) audio datawith embedded camera control instructions, and 2) USBX data for the USBsystem.

[0204] Conversely, the spectrum for user interface signals transmittedfrom the blade computer 102 may be divided thus: audio signals may betransmitted in the frequency range of approximately 100 KHz throughapproximately 170 KHz; and USB signals may be transmitted in frequenciesfrom approximately 400 KHz upwards. This spectral allocation may beperformed using a number of high-, low-, and band-pass filters, asillustrated in FIGS. 20 and 21, described below.

[0205] FIGS. 20-21—Processing of Multi-Media Human Interface Signals

[0206]FIGS. 20 and 21 are block diagrams of signal filtering systemsused by the communications processor 804 to transmit human interfacesignals between the human interface 130 and the blade computer 102according to the spectrum allocation schemes described above withreference to FIG. 19.

[0207]FIG. 20 is a block diagram of a filtering system on the humaninterface side of the distribution system. In the filtering system ofFIG. 20, human interface signals may be received from the various humaninterface devices included in the human interface 130, such as, forexample, camera 135, microphone 2008, etc. As mentioned above, in oneembodiment, user interface signals transmitted to and from a given humaninterface 130 may be passed through an extender device 121B, e.g., anencoding/decoding device which in some embodiments may be referred to asa C/Port. For example, USB signals to and from a mouse, keyboard, etc.,of the human interface 130 may be routed through the extender device121B, as shown.

[0208] For example, in one embodiment, the remote extender device 121B,e.g., the C/Port, may receive RGB output from the camera 135, e.g., adigital camera-on-a-chip, and convert the RGB signals to an (analog)NTSC signal with a frequency less than 2.5 MHz. As FIG. 20 shows, theNTSC signal may then be added to a USB return line from the extenderdevice 121B by summer 2075A (after passing through high-pass filter2030B), and transmitted to the blade computer 102 as USB signals 2090,where the camera signals may then be isolated from the USB carrier anddata by band pass limiting filtering. In one embodiment, the USB signalstransmitted between the extender device 121B and the blade switch 809associated with the human interface 130 may be extended USB (USBX)signals with the spectrum allocation described above. For furtherdetails regarding extended USB signaling, please see U.S. patentapplication Ser. No. 09/619,989 titled “System And Method For ProvidingA Remote Universal Serial Bus”, which was incorporated by referenceabove.

[0209] As FIG. 20 also shows, in one embodiment, signals may be receivedfrom a microphone 2008, passed through an automatic gain control (AGC)2060, multiplexed with a 5 MHz carrier frequency 2070, passed through aband-pass filter 2050A, and combined with the converted NTSC camerasignals for transmission to the blade computer 102, as shown.

[0210] Thus, signals transmitted from the HI to the blade computer 102may include the high frequency USBX signal, a low frequency camera datasignal, and a mid-band audio modulated channel. As the audio may be theresult of a microphone input the dynamic range needed may exceed thelimits of the modulation scheme, and so the audio may be run though anAGC (Automatic Gain Control) to assure a proper signal level. The audiosignal may be modulated on the carrier signal and mixed in with thebase-band NTSC video signal. Energy above the video and audio range maybe filtered out and the resulting combination signal sent upstream tothe blade computer 102.

[0211] Thus, as FIG. 20 shows, human interface signals may be receivedfrom the human interface devices and passed through respective filtersbefore being transmitted to the respective blade computer 102.

[0212] Conversely, human interface signals may also be sent from theblade computer 102 to various human interface devices, e.g.,earpiece/headphone 2002 and other devices coupled to the remote extenderdevice 121B, e.g., monitors, speakers, etc., as shown. For example, USBsignals 2090 from the blade computer 102 may include audio signalstargeted at the earpiece 2002, and command signals targeted to thecamera 135 (e.g., specifying image size and/or resolution), as mentionedabove, as well as USB human interface signals targeted to the HI viaextender device 121B. The audio and command signals may be separated outvia low-pass filter 2020A, while the USB signals may be passed throughhigh-pass filter 2030A and transmitted to the extender device 121B fordistribution to respective USB human interface devices. The commandsignals may be separated from the audio signals and transmitted to thecamera 135, as shown.

[0213] The human interface signals from the blade USB 2090 targeted forthe extender device 121B may be separated from the audio and commandsignals by the high-pass filter 2030A and transmitted to the extenderdevice 121B as shown.

[0214]FIG. 21 is a block diagram of the filtering system on the bladecomputer side of the distribution system, according to one embodiment.More specifically, in one embodiment, the filtering system shown in FIG.21 may be included in the host extender device 121A coupling the bladecomputer 102 to the communication processor 804.

[0215] As FIG. 21 shows, in this embodiment, user interface signalsoriginating from the blade computer 102 targeted to the human interface130 (e.g., the remote extender device 121B) may include audio signals2108A, command data 2180, and USB interface device signals 2104A. In oneembodiment, the command data 2180 may be processed by a frequency shiftkey (FSK) module 2150 (e.g., at 20 KHz), then merged or combined withthe audio signals 2108A via summer 2075C. In one embodiment, the commanddata or signals 2180 may have a frequency higher than the human ear candetect (and possibly higher than the earpiece, headphones, or speakers,can effectively play), and the audio signals may be at frequencies lowerthan the command signals, such that the camera 135 may not respond tothe audio signals, while responding to the command signals. In oneembodiment, the camera 135 may include a detector which utilizesfrequency shift keying for detecting commands, listening for a controltone (the command signal) and noting when it changes frequency, e.g.,where shifting from 20 kHz to 19 kHz and back to 20 kHz represents onebit. Thus, the carrier for the FSK 2150 is preferably above the audioband.

[0216] The combined audio/command signals may then be filtered throughlow-pass filter 2020C, e.g., to remove any energy that would effect theUSB signal, and transmitted to summer 2075B. USB human interface signals2104B from the blade computer 102 targeted for the HI, e.g., the remoteextender device 121B, i.e., USB to HI signals 2104B, may be filtered byhigh-pass filter 2030C, e.g., to remove any energy that might interferewith the audio signals, and transmitted to summer 2075B, where they maybe combined with the audio/command signals and transmitted to theextender device 121B, e.g., the HI 130, as indicated by the USB to HIsignal path 2104A. In one embodiment, the USB to HI signals 2104B mayinclude a 6 MHz carrier with modulation.

[0217] In the embodiment shown, user interface signals originating fromthe human interface 130 (e.g., from the extender device 121B) targetedfor the blade computer 102 may include USB from HI signals 2114, audiosignals 2108B, and video signals 2135. Combined human interface signalsmay be received from the extender device 121B and split out into theaudio signals 2108B, the video signals 2135, and the USB from HI signals2114 by band-pass filter 2050B, low-pass filter 2020D, and high-passfilter 2030D, respectively. In one embodiment, the audio signal 2108 mayalso pass through a detector 2160, as shown. Thus, filters (2030D,2050B, and 2020D) may disassemble the return signal from the remoteextender device 121B at the HI 130, e.g., the C/Port, starting with thehigh frequency filter 2030D extracting the USBX data stream fortransmittal to the blade computer 102. Band-pass filter 2050B mayseparate return audio data 2108 from the carrier (e.g., a 5 MHzcarrier), which may be detected and fed to the audio system (of theblade computer 102). Finally, a low-pass filter may recover thebase-band video 2135 which may be fed to a subsequent NTSC decoder andused as needed. Thus, the filtering system of FIG. 21 may “undo” thesignal combining performed by the system of FIG. 20.

[0218] Various embodiments of the above described system may thusprovide means for transporting streaming real-time video back from theuser's desktop to the computer 102 without packetization. Additionally,a separate (non-USB) audio link may be established in both directionswhich uses packetization of the data. Finally, control information maybe sent to the camera to control such features as resolution andpan/tilt positioning commands, among others.

[0219]FIG. 22—Communication Video and Audio Signal Delivery to a VideoBus

[0220]FIG. 22 is a block diagram illustrating the delivery of video andaudio signals onto the video bus 810, according to one embodiment. AsFIG. 22 shows, controller 2200 may be operable to send commands 2280targeted to the camera 135, e.g., specifying camera image resolution,size, etc. The controller 2200 may also be operable to send controlsignals to the NTSC-to-Y/C converter 2210, memory 1106, D/A converter1103, and video switch 808.

[0221] As FIG. 22 shows, in one embodiment, audio signals 2108 may betransmitted bi-directionally (Bi-Di) through current transceiver 2260,which may communicate these audio signals bi-directionally with aselected VBus 810 via the video switch 808. In other words, audiosignals may be current modulated onto and current demodulated from thecommand line of the selected VBus 810 using a bi-directional currentsource.

[0222] As FIG. 22 also shows, incoming NTSC video signals 2135, e.g., acamera image of a video conference participant, may first be convertedto Y/C format (e.g., 12-bit digital) by NTSC-to-Y/C converter 2210. Theconverted signals may then be stored in memory 1106, as shown. When readfrom memory 1106 (by controller 2200), the digital signals may beconverted to analog by D/A converter 1103, as shown. Finally, theconverted analog video signals (Y/C) may be placed on the selected VBus810 as a sub-frame.

[0223]FIG. 23—System Control

[0224]FIG. 23 illustrates system control for the communications andcontent distribution system, according to one embodiment. As the systemdepends on many elements performing assigned tasks which may varyconsiderably, some type of overall control may be necessary. In oneembodiment, the user of the system may control the system through asemi-autonomous user interface, i.e., where the user interface isoperated at least partially independent from the central system. Forexample, the user interface may be implemented as a web page.Alternatively, the user interface may implemented as an executable (orinterpretable) program that runs on one or more of the computing systems102. The user may provide input to the interface indicating tasks to beperformed, configurations to be set, and/or decisions to be made. Thetask, configuration, and/or decision may then be implemented in and bythe system via a series of simple instructions which may program theelements appropriately.

[0225] In the embodiment shown, a command computer 2302 may couple tothe content processor 806 via content processor command line 2320, andto the communications processor 804 via primary control bus 920, asshown. A plurality of blade VBus switches 809 may provide switchingfunctionality for the VBuses 810 in response to commands received fromthe command computer 2302 over the primary control bus 920.

[0226] In the control system illustrated in FIG. 23, a main controllermay be included in the content processor 806. In one embodiment, many ofthe features of the communications processor 804 may be distributedthroughout the system, and thus the communications processor 804 may beconsidered a virtual device that is also included in the maincontroller. The communications distribution system may interface to theoutside world via a hardware process included on the video buses 810.

[0227] As described above, the content processor 806 may receive videosignals (i.e., content) from the various content sources 860, decode thecontent, and route the decoded content to one or more of the video buses810. The communications interface may be operable to format VBuses 810for analog and/or IP connection to other networks, e.g., IP, standardvideo protocols such as RGB, Y/C, etc., or other communication protocolsand techniques as described in one or more of the U.S. patents andpatent applications incorporated by reference above.

[0228] Content Operations

[0229] In one embodiment, the command computer 2302 may communicatecontrol information to the content processor 806 specifying what contentsignals are to be carried on any specific VBus 810. In the embodimentshown, these control commands may be transmitted on the contentprocessor command line 2320. Generally, a VBus 810 may be committed toeither an image from the content sources 860, or to be a link forcommunications frames. In the case where the VBus 810 is used to carrycontent images, both the particular content source 860 and a sourcedecoding scheme is preferably specified along with the actual VBus to beused for the content distribution. When the VBus 810 is used todistribute communications, the VBus 810 may simply be left alone, e.g.,unloaded and signal free.

[0230] The primary control bus 920, shown in FIG. 23, may be used tonotify the individual blade VBus switches 809 as to which VBus to switchto for content distribution. Additionally, the command computer 2302 maysignal the individual blade VBus switches 809 over the command line ofeach VBus 810 as to the size and location of the images inserted on theuser's screen, e.g., upper left and lower right corners of the imagerelative to the user's screen size, as described above. A blade VBusswitch 809 may be operable to receive multiple instructions describinginsertion for respective multiple images. In other words, in someembodiments, the blade VBus switch 809 may support multiple imageinsertions, where a plurality of images are displayed at the same timeon the user's monitor screen.

[0231] Communications Operations

[0232] In one embodiment, when a communications session, e.g., a videoconference session, is initiated or activated one or more of thefollowing commands may be issued by the command computer 2302:

[0233] The content processor 806 may be instructed to clear a particularVBus 810 and specify it (the VBus) as a communications session VBus. Thecontent processor 806 may then place an “RS” command and associated dataonto the specified VBus, thereby providing a timing base for theinsertion of video images (e.g., communications video signals). This RScommand may be transmitted to the content processor 806 over the contentprocessor command line 2320.

[0234] Each of the blade computers participating in the communicationssession may be instructed as to which VBus to switch to for the session.These instructions may be transmitted over the primary control bus 920.

[0235] Session control software, e.g., executing on the command computer2302 or on the participating blade computers 102, may compute the sizeand location of each participant's image on each of the otherparticipant's screens. This information, possibly along with sequencing,traffic rules, resolution commands for the camera, and/or rate commentsfor camera image scaling systems may be transferred to the respectiveblade computers 102 over the primary control bus 920.

[0236] The implementation of the above commands may thus result inactivation of the communications session.

[0237]FIG. 24—Method for Insertion of Video Content and Communications

[0238]FIG. 24 is a flowchart of one embodiment of a method for insertingvideo content and/or video communications into video signals for displayin a human interface 130, e.g., a computer monitor 116 or other displaydevice of the human interface 130, as may be performed, for example, bythe system described above with reference to FIG. 11. It is noted thatthe embodiment of the method described below is meant to be exemplaryonly, and is not intended to limit the invention to any particularapproach. For example, in various embodiments, two or more of the stepsdescribed may be performed concurrently or in a different order thanshown, or may be omitted. Additional steps may also be performed.Additionally, as noted above, although in the embodiment describedbelow, the video data comprises analog video signals, other embodimentsare also contemplated where the video data comprises digital video dataor a combination of both analog and digital video data.

[0239] As FIG. 24 shows, in 2402, first analog video signalscorresponding to a first image may be received from a first computingsystem 102, e.g., a co-located computing system 102, where the firstanalog video signals are intended for display by a first (possiblyremote) human interface 130A. For example, the video signals may includea typical desktop image for a computer system. As described in detailabove, in one embodiment, the received video signals may originate fromany of the plurality of co-located computers in a communications and/orcontent distribution system.

[0240] In 2404, a second image may be received. For example, asdescribed above, the second image may be received from a camera in asecond (possibly remote) human interface 130B, such as in a videoconferencing or video telephony session, where a plurality of users mayeach send and receive camera images (and accompanying audio) of eachother. In other embodiments, the second image may include videocommunications, e.g., or video content originating from a variety ofsources, such as television tuners, satellite tuners, cable tuners,digital video sources, etc.

[0241] In 2406, second analog video signals corresponding to the secondimage may be inserted into the first analog video signals to generatethird video signals. For example, in the video conferencing examplegiven above, respective video images may be inserted into each of theother participant's screen images, allowing the participants to view(and optionally hear) one another via their respective human interfaces.Additionally, media content, such as television broadcasts, may also beinserted into the screen images, allowing each participant to viewrelated video content during the video conference. As another example, astreaming news broadcast may be inserted into the screen image of amonitor 116 (in a human interface 130) as a Picture-in-Picture (PIP)image, allowing the user of the human interface 130 to view (andpossibly hear) relevant media content while working.

[0242] In one embodiment, the second analog video signals may begenerated from a modified version of the second image. For example, thesecond image may be scaled, e.g., reduced in size and/or resolution, orotherwise processed for display on the human interface display device116, as indicated above. Thus, the third video signals may include atleast a portion of the original (first) image plus at least a portion ofthe second image, e.g., in a PIP format.

[0243] In 2408, the third video signals may be transmitted to the firsthuman interface 130A, e.g., over a serial bus 110. As described indetail above, transmitting the third video signals may involve the useof signal transmission extender technologies, such as, for example,encoding the signals into the USBX protocol prior to transmission.

[0244] Finally, in 2410, an image may be displayed on the display device116 of the first human interface 130A, based on the received third videosignals. For example, a PIP image based on the first and second videoimages may be displayed on the monitor 116 of the first human interface.It should be noted that although the method was described in terms ofinserting a second image into a first image, the described technique maybe used to insert a plurality of video images from a variety of sourcesinto the first image. Thus, video images from multiple sources,including the user's computer system 102, may be combined in real-timeto provide video content and/or video communications to a user of thesystem.

[0245]FIGS. 25 and 26—Methods for Communication Between a Computer and aRemote Human Interface

[0246]FIGS. 25 and 26 flowcharts embodiments of methods forcommunications between a computer 102 and a remote human interface (RHI)130. More specifically, the methods of FIGS. 25 and 26 illustrate anapproach whereby human interface signals, e.g., audio, video, and/or I/Odevice signals (e.g., USB signals), may be transmitted in bothdirections over a single transmission medium, e.g., serial link 110,according to one embodiment. In a preferred embodiment, the methodsutilize the system illustrated in FIGS. 19-21, described above. It isnoted that the methods presented in FIGS. 25 and 26 may be used inconjunction with one another to facilitate the two-way communicationbetween the computer 102 and the remote HI 130. It is further noted thatembodiments of the methods of FIGS. 25 and 26 may be used to implementvarious embodiments of the content and/or communications distributionsystems described above.

[0247]FIG. 25—Method for Transmitting HI signals from the Computer tothe Remote HI

[0248]FIG. 25 flowcharts one embodiment of a method for communicationsbetween a computer 102 and a remote human interface 130. Morespecifically, the method of FIG. 25 illustrates an approach wherebyaudio and I/O device signals (e.g., USB signals), may be transmittedfrom the computer 102 to the remote human interface 130, e.g., to aplurality of HI devices included in the HI 130. In a preferredembodiment, the method utilizes the system illustrated in FIGS. 19-21,described above. It should be noted that in various embodiments, some ofthe steps described may be performed concurrently, or in a differentorder than shown, or may be omitted. Additional steps may also beperformed.

[0249] As FIG. 25 shows, in 2502, a plurality of host human interface(HHI) signals may be received from a computer 102, where each of theplurality of HHI signals is targeted to a corresponding HI deviceincluded in the remote human interface 130. Note that as used herein,the term “host human interface signals” refers to signals originatingfrom the host computer and/or devices included in the host computer 102,which are to be sent to the remote human interface 130.

[0250] In a preferred embodiment, the plurality of HHI signals includean audio signal 2108A and one or more I/O device signals 2104B. Forexample, the audio signal 2108A may be targeted to speakers, headphone,or earpiece of the HI 130. Similarly, the one or more I/O device signals2104B may be targeted to a respective one or more I/O devices of the HI130. In one embodiment, the I/O devices may include a computer keyboard,a mouse or other pointing device, a PDA IR device, or any other I/Odevice as desired. In one embodiment, the one or more remote I/O devicesignals may be extended USB (USBX) signals, and the one or more I/Odevices included in the HI may include one or more USB devices, althoughother protocols and other types of I/O devices are also contemplated. Asdescribed above, in a preferred embodiment, the HI 130 may also includea video camera 135 which may operate to generate a video stream, e.g.,of user images, and which may be configured by a command signal from thecomputer 102. As also described above, in one embodiment, the HHIsignals may be received by a host encoder/decoder, e.g., a host extenderdevice 121A, which may include at least a portion of the systemsdescribed with reference to FIGS. 20 and 21.

[0251] Once the HHI signals are received in 2502, then in 2504, theplurality of HHI signals may be combined into a combined HHI signal. Inother words, the audio 2108A and I/O device signals 2104B may becombined into a combined signal for transmission over a singletransmission medium, e.g., serial link 110, to the HI 130. For example,referring back to FIGS. 19-21, the method may utilize various filters,frequency shifters, etc., to populate respective portions of theavailable spectrum of the transmission medium 110 with the various HIsignals.

[0252] In one embodiment, command signals for the video camera 135 mayalso be included in the combined HHI signal. For example, as describedabove with reference to FIG. 21, in an embodiment where the plurality ofHHI signals include a command signal for the video camera 135 includedin the HI, combining the plurality of HHI signals into a combined HHIsignal may include combining the audio signal and the command signalinto a combined audio/command signal, e.g., via summer 2075C, and thencombining the combined audio/command signal and the one or more I/Odevice signals into the combined HHI signal, e.g., via summer 2075B. Inone embodiment, a frequency shift key (FSK) encoder module 2150 mayprocess the command signal prior to combining the audio signal and thecommand signal into the combined audio/command signal, e.g., to encodethe command signal in a frequency range which is inaudible to the humanear. In the embodiment shown in FIG. 21, the 20 kHz FSK 2150 may operateto shift the command signal frequency to the 20 kHz range, where, forexample, a carrier signal of 20 kHz shifts to 19 kHz and back to 20 kHzto indicate a bit (e.g., a “1”). In other words, the command signal 2180may be switched between 20 kHz and 19 kHz to encode a sequence of onesand zeros which may operate to configure the video camera 135 of the HI130. It is noted that a corresponding FSK decoder, e.g., coupled to orincluded in the camera 135, may operate on the remote HI end to decodethe command signal back to digital commands (ones and zeros) for thecamera 135.

[0253] As indicated in FIG. 21, in one embodiment, a low-pass filter2020C may be applied to the combined audio/command signal prior tocombining the combined audio/command signal and the one or more I/Odevice signals into the combined HHI signal, e.g., to remove any signalenergy (e.g., frequencies) that might interfere with the I/O devicesignals, e.g., USBX signals.

[0254] Similarly, in one embodiment, a high-pass filter 2030C may beapplied to the one or more I/O device signals prior to combining thecombined audio/command signal and the one or more I/O device signalsinto the combined HHI signal, e.g., to remove any signal energy (e.g.,frequencies) that might interfere with the audio/command signal.

[0255] Once the HHI signals have been combined into a combined HHIsignal, then in 2506, the combined HHI signal may be transmitted over atransmission medium, e.g., to the remote HI 130, e.g., to a remoteextender device 121B, as described above. It should be noted that in apreferred embodiment, prior to applying the high-pass filter, the one ormore I/O device signals 2104B may be encoded using distance extensiontechnology, e.g., may be encoded into USBX signals for transmission overdistances longer than typically allowable by standard transmissionprotocols, such as USB. For further information regarding extensiontechnologies, please see U.S. patent application Ser. No. 09/619,989titled “System And Method For Providing A Remote Universal Serial Bus”,which was incorporated by reference above.

[0256] The combined HHI signal may then be received over thetransmission medium, as indicated in 2508, e.g., by the system describedabove with reference to FIG. 20. For example, in one embodiment, aremote encoder/decoder included in the remote HI 130 may include thesystem of FIG. 20, and may receive the combined HHI signal, as describedabove.

[0257] In 2510, the audio signal and the one or more I/O device signalsmay be extracted from the combined HHI signal. In one embodiment,extracting the audio signal and the one or more I/O device signals fromthe combined HHI signal may include applying a low-pass filter 2020A tothe combined HHI signal to generate the audio signal, as illustrated inFIG. 20. Similarly, as also shown in FIG. 20, in one embodiment, ahigh-pass filter 2030A may be applied to the combined HHI signal togenerate the one or more I/O device signals.

[0258] In an embodiment where a command signal is included with theaudio signal, e.g., where the combined HHI signal includes a combinedaudio/command signal, the low-pass filter may be applied to the combinedHHI signal to generate the combined audio/command signal including theaudio signal and the command signal targeted to the video camera of thehuman interface.

[0259] Finally, as indicated in 2512, the audio signal may betransmitted to an audio device included in the HI, e.g., to an earpiece2002, as shown in FIG. 20, or to any other type of audio device, such asspeakers, headphone, and so forth. In an embodiment where the combinedHHI signal includes a combined audio/command signal, the command signalmay be transmitted to the video camera of the human interface, where thecommand signal operates to configure the video camera, e.g., the commandsignal may operate to set a resolution level of the video camera. Asmentioned above, in one embodiment, an FSK decoder may be coupled to orincluded in the camera 135, and may decode the command signal back todigital commands (ones and zeros) for the camera 135.

[0260] In a preferred embodiment, due to their respective frequencies,the audio signal may not have to be separated from the command signalbefore their respective propagations to the audio device and the videocamera. In other words, if the command signal has a frequency rangewhich is above (or below) the audible frequency range for humans, thenthe combined audio/command signal may be sent to the audio device as is,and the inaudible command signal may simply be ignored. Similarly, aslong as the frequency range of the audio signal does not interfere withthe command signal, the combined audio/command signal may be sent to thevideo camera 135, where the command signal portion may be used toconfigure or control the camera, and the audio signal may be ignored. Ofcourse, if desired, in one embodiment, the audio signal and the commandsignal may each be extracted from the combined audio/command signal,e.g., by low-, band-, or high-pass filters, and the respective signalssent to their respective devices. In other words, the audio signal maybe separated from the command signal prior to transmitting the signalsto the audio device and the video camera, respectively.

[0261] Similarly, the one or more I/O device signals may be transmittedto respective one or more I/O devices included in the HI 130, e.g., aUSB keyboard, USB mouse, etc., as was described in detail above. In anembodiment where the one or more I/O device signals were encoded usingan extension technology, e.g., where the I/O device signals were encodedinto USBX signals, the USBX I/O device signals may be decoded prior totransmittal to the respective I/O devices. For example, the USBX I/Odevice signals may be decoded to standard USB signals, then transmittedto respective USB devices.

[0262] Thus, various embodiments of the method described above mayoperate to communicate human interface signals from the computer to theremote human interface. Additionally, I/O device signals may beencoded/decoded to and from a special extension protocol (e.g., USBX)that allows the signals to be communicated over substantially longerdistances than typically allowed by I/O device signal transmissionprotocols, e.g., USB.

[0263] One embodiment of a method for transmitting HI signals in theother directions, e.g., from the remote HI 130 to the computer, isdescribed below with reference to FIG. 26. As mentioned above, in apreferred embodiment, the method of FIG. 25 may operate in conjunctionwith that of FIG. 26 to facilitate two-way HI signal communicationsbetween the computer 102 and the remote HI 130 over a singletransmission medium, e.g., over a single cable 110.

[0264]FIG. 26—Method for Transmitting HI signals from the Remote HI tothe Computer

[0265]FIG. 26 flowcharts one embodiment of a method for communicatinghuman interface signals from the remote human interface 130 to thecomputer 102. More specifically, the method of FIG. 25 illustrates anapproach whereby audio, video, and I/O device signals (e.g., USBsignals), from a plurality of HI devices may be transmitted from theremote human interface 130 to the computer 102. In a preferredembodiment, the method utilizes the system illustrated in FIGS. 19-21,described above. It should be noted that in various embodiments, some ofthe steps described may be performed concurrently, or in a differentorder than shown, or may be omitted. Additional steps may also beperformed.

[0266] As FIG. 26 shows, in 2602, a plurality of remote human interface(RHI) signals may be received from a respective plurality of humaninterface devices include in the remote human interface 130. In oneembodiment, the plurality of human interface devices included in theremote human interface 130 include a microphone or other audio signalgenerator, a video camera 135, and one or more remote I/O devices, e.g.,keyboard, mouse, telephone, etc. Thus, the plurality of RHI signals mayinclude an audio signal from the microphone of the remote HI, a videosignal from the video camera 135 in the remote HI, and one or moreremote I/O device signals from the one or more I/O devices in the remoteHI. As noted above, any other HI devices or peripherals are alsocontemplated as falling within the scope of the present systems andmethods.

[0267] As noted above, in one embodiment, the one or more remote I/Odevice signals may be extended USB (USBX) signals, and the one or moreI/O devices included in the HI may include one or more USB devices,although other protocols and other types of I/O devices are alsocontemplated. As described above, in a preferred embodiment, the HI 130may also include video camera 135 which may operate to generate a videostream, e.g., of user images. As also described above, in oneembodiment, the RHI signals may be received by a remote encoder/decoder,e.g., a remote extender device 121B, which may include at least aportion of the systems described with reference to FIGS. 20 and 21.

[0268] In 2604, the plurality of RHI signals may be combined into acombined RHI signal. In other words, the audio, video, and I/O devicesignals may be combined into a combined RHI signal for transmission overa single transmission medium (serial link 110) to the computer 102. Forexample, referring back to FIGS. 19-21, the method may utilize variousfilters, frequency shifters, etc., to populate respective portions ofthe available spectrum of the transmission medium 110 with the variousHI signals.

[0269] For example, as FIG. 20 shows, in one embodiment, combining theplurality of RHI signals into a combined RHI signal may includecombining the audio signal and the video signal into a combinedaudio/video signal, and combining the combined audio/video signal andthe one or more I/O device signals into the combined RHI signal. As FIG.20 also shows, in one embodiment, a low-pass filter 2020B may be appliedto the combined audio/video signal prior to combining the combinedaudio/video signal and the one or more I/O device signals into thecombined RHI signal. Similarly, a high-pass filter 2030B may be appliedto the one or more remote 1/0 device signals prior to combining thecombined audio/video signal and the one or more remote I/O devicesignals into the combined HI signal. These filters may be applied to therespective signals to remove energy (frequencies) that may interferewith the other signals, as was mentioned above.

[0270] In some embodiments, the audio signal from the microphone 2008may not provide a reliable signal level, and thus, prior to combiningthe audio signal and the video signal into the combined audio/videosignal, an automatic gain control may be applied to the audio signal, asshown in FIG. 20. Referring back to FIG. 19, in a preferred embodiment,the audio signal 1920 from the microphone may be transmitted in adifferent frequency range than that of the original audio signalproduced by the microphone, thus, in one embodiment, the automatic gaincontrolled audio signal may be modulated onto a carrier signal, e.g., a5 MHz, for example, by MUX 2070, thereby effectively shifting thefrequency of the audio signal into the desired frequency range. Themodulated audio signal may then be passed through a band-pass filter2050A, as shown. Thus, in this embodiment, combining the audio signaland the video signal into the combined audio/video signal includescombining the modulated audio signal and the video signal into thecombined audio/video signal.

[0271] In one embodiment, the video signal received from the videocamera 135 may include an RGB signal, as is well known in the art.However, it may be desirable to transmit the video signal in a differentformat, such as NTSC, for example. Thus, in one embodiment, prior tocombining the audio signal and the video signal into the combinedaudio/video signal, the video signal may be converted to an NTSC videosignal, e.g., by the RGB-to-NTSC converter 2040 shown in FIG. 20. It isnoted that other video protocols (besides NTSC) are also contemplated.

[0272] The combined RHI signal may then be transmitted over thetransmission medium 110, e.g., to the host extender device(encoder/decoder) 121A, as indicated in 2606. Note that in a preferredembodiment, the transmission medium is the serial link 110 used in themethod of FIG. 25 to transmit HHI signals from the computer 102 to theremote HI 130. In other words, the serial link 110 may be used tocommunicate both ways between the computer 102 and the remote HI 130.

[0273] In 2608, the combined RHI signal may be received over thetransmission medium, e.g., by the system of FIG. 21 or its equivalent.For example, in one embodiment, the system of FIG. 21 may be included inor coupled to the host extender device 121A, which was described indetail above. The audio signal, the video signal, and the one or moreI/O device signals from the one or more I/O devices may then beextracted from the combined RHI signal, as indicated in 2610.

[0274] For example, referring again to FIG. 21, in one embodiment,extracting the audio signal, the video signal, and the one or moreremote I/O device signals from the combined RHI signal may includeapplying a high-pass filter 2030D to the combined RHI signal to generatethe one or more remote I/O device signals. Similarly, a low-pass filter2020C may be applied to the combined HI signal to generate the videosignal 2135. Furthermore, in an embodiment where the generated videosignal includes an NTSC video signal, the generated video signal may besent to an NTSC decoder which may decode the NTSC video signal, e.g.,back to an RGB video signal. In one embodiment, a band-pass filter 2050Bmay be applied to the combined HI signal to generate the audio signal2108B. In a further embodiment, the generated audio signal may be sentthrough a detector 2160 to extract audio data. In other words, thedetector 2160 may operate to determine whether audio signals or data areincluded in the filtered signal, and to extract the determined audiodata for provision to the computer 102. In various embodiments, thedetector, or a separate audio processor, may operate to perform any ofvarious conversions or processing on the audio signal or data inaccordance with desired audio formats.

[0275] Finally, in 2612, the extracted audio signal, the extracted videosignal, and the extracted one or more I/O device signals may betransmitted to the computer 102. In an embodiment where the extractedvideo signal was decoded from NTSC, the decoded video signal may be sentto the computer. Similarly, in an embodiment where the audio data wasextracted from the audio signal, e.g., by the detector 2160, theextracted audio data may be sent to the computer.

[0276] As noted above, in one embodiment, the (remote) I/O devicesignals may be formatted in accordance with an extension protocol, andthus, once the I/O device signals have been extracted from the combinedsignal, the extracted I/O device signals may be passed through adecoder, e.g., a remote extender 120B, to convert the signals back to astandard I/O device transmission protocol. For example, in a preferredembodiment, the one or more remote I/O device signals may be extendedUSB (USBX) signals, and the one or more I/O devices included in theremote HI may include one or more USB devices. In this embodiment, theUSBX I/O device signals may be sent to a USBX decoder, e.g., the remoteextender 120B, which may convert the USBX signals to USB signals whichmay then be transmitted to the computer 102.

[0277] Thus, various embodiments of the method described above mayoperate to communicate human interface signals from the remote humaninterface 130 to the computer 102. Additionally, I/O device signals maybe encoded/decoded to and from a special extension protocol (e.g., USBX)that allows the signals to be communicated over substantially longerdistances than typically allowed by I/O device signal transmissionprotocols, e.g., USB.

[0278] As mentioned above, in a preferred embodiment, the method of FIG.26 may operate in conjunction with that of FIG. 25 to facilitate two-wayHI signal communications between the computer 102 and the remote HI 130over a single transmission medium, e.g., over serial cable 110.

[0279] Various embodiments further include receiving or storinginstructions and/or data implemented in accordance with the foregoingdescription upon a carrier medium. Suitable carrier media include amemory medium as described above, as well as signals such as electrical,electromagnetic, or digital signals, conveyed via a communication mediumsuch as networks and/or a wireless link.

[0280] Further modifications and alternative embodiments of variousaspects of the invention will be apparent to those skilled in the art inview of this description. Accordingly, this description is to beconstrued as illustrative only and is for the purpose of teaching thoseskilled in the art the general manner of carrying out the invention. Itis to be understood that the forms of the invention shown and describedherein are to be taken as the presently preferred embodiments. Elementsand materials may be substituted for those illustrated and describedherein, parts and processes may be reversed, and certain features of theinvention may be utilized independently, all as would be apparent to oneskilled in the art after having the benefit of this description of theinvention. Changes may be made in the elements described herein withoutdeparting from the spirit and scope of the invention as described in thefollowing claims.

We claim:
 1. A system comprising a plurality of computing systemssystem, the system comprising: a plurality of computing systems, whereinthe plurality of computing systems are located at a common location; aplurality of human interfaces, wherein each of the human interfaces islocated remotely from the common location, wherein each of the humaninterfaces includes a display device for displaying images, and whereina first computing system is operable to generate a first image to bedisplayed on a first display device in a respective first humaninterface; a plurality of communication mediums coupling each of theplurality of computing systems to at least one of the plurality of humaninterfaces; and a video manager coupled to each of the plurality ofcomputing systems, wherein the video manager is operable to selectivelyinsert a second image into the first image generated by the firstcomputing system for display on the first display device.
 2. The systemof claim 1, wherein the first display device is operable to display thefirst image including the second image.
 3. The system of claim 2,wherein, in being operable to display the first image including thesecond image, the first display device is operable to display a PIP(Picture-In-Picture) image including at least a portion of the firstimage and at least a portion of the second image.
 4. The system of claim1, wherein the second image is received from a content source.
 5. Thesystem of claim 4, wherein the second image is received from an externalimage source over a network.
 6. The system of claim 4, wherein the firstimage comprises first analog video signals, and wherein the second imagereceived from the content source comprises second analog video signalsin a scan line format.
 7. The system of claim 6 wherein the videomanager is operable to: store the second image in the memory medium;receive the first analog video signals from the first computing system;retrieve the stored second image from the memory medium in digital form;convert the second image from digital form to the second analog videosignals; and insert the second analog video signals into the firstanalog video signals on a scan line basis, thereby generating the thirdvideo signals. 8 The system of claim 6 wherein each of at least a subsetof the plurality of human interfaces includes an audio device, whereinthe video manager is further operable to: receive audio signalscorresponding to the first analog video signals; and transmit the audiosignals with the third analog video signals to the first human interfacefor presentation to a user of the first human interface.
 9. The systemof claim 4, wherein the video manager is further operable to: modifyformat, protocol, size, and/or resolution of the second image fordisplay on the display device of the first human interface
 10. Thesystem of claim 1, wherein the second image is received from acommunication source.
 11. The system of claim 10, wherein each of thehuman interfaces includes a camera for acquiring an image.
 12. Thesystem of claim 11, wherein the second image is received from a cameracomprised in a second human interface of the plurality of humaninterfaces.
 13. The system of claim 11, wherein the second imagecomprises a videoconferencing image of a user of another one of thecomputing systems.
 14. The system of claim 11, wherein the video manageris operable to selectively insert a plurality of second images into thefirst image generated by the first computing system for display on thefirst display device.
 15. The system of claim 14, wherein the pluralityof second images comprise video-conferencing images of users of otherones of the computing systems.
 16. The system of claim 10, wherein thesecond image is received from an external camera over a network.
 17. Thesystem of claim 1, wherein the first computing system is operable togenerate a sequence of first images to be displayed on the first displaydevice in the respective first human interface; wherein the videomanager is operable to selectively insert a sequence of second imagesinto the sequence of first images generated by the first computingsystem for display on the first display device, wherein each secondimage of the sequence of second images is inserted into a respective oneof the sequence of first images.
 18. The system of claim 1, wherein thevideo manager comprises a frame buffer; and wherein the video manager isfurther operable to: store the received second image in the framebuffer, wherein the frame buffer corresponds to a display screen of thedisplay device of the first human interface; and retrieve the secondimage from the frame buffer for insertion into the first image.
 19. Thesystem of claim 1, wherein the video manager is further operable to:modify format, protocol, size, and/or resolution of the second image fordisplay on the display device of the first human interface.
 20. Thesystem of claim 1, wherein each of the plurality of computing systemscomprises a computer-on-a-card.
 21. The system of claim 1, wherein eachof the plurality of computing systems comprises a computer blade. 22.The system of claim 21, wherein the plurality of computing systems arecomprised in one or more rack-mount structures.
 23. The system of claim1, where at least one of the plurality of human interfaces comprises aplurality of display devices.
 24. The system of claim 23, wherein eachof the plurality of display devices comprises a computer monitor. 25.The system of claim 1, wherein the video manager is operable toselectively insert the second image into the first image without usingany CPU cycles of the computing systems.
 26. The system of claim 1,wherein the second image is received from a content source.
 27. A videomanager, comprising: a video switch comprising: a memory medium; and aprocessor coupled to the memory medium; and a content processor coupledto the video switch through one or more video buses; wherein the videoswitch is operable to couple to a plurality of co-located computingsystems, and to further couple to a plurality of human interfaces,wherein each of the human interfaces is located remotely from theco-located computing systems, wherein each of the human interfacesincludes a display device for displaying images; wherein the videoswitch is further operable to: receive a first image generated by afirst computing system to be displayed on a first display device in arespective first human interface, wherein the first image comprisesfirst analog video signals in a scan line format; receive a second imagefrom the content processor, wherein the second image comprises secondanalog video signals in the scan line format; and selectively insert thesecond image into the first image generated by the first computingsystem for display on the first display device by inserting the secondanalog video signals into the first analog video signals.
 28. The videomanager of claim 27, wherein the video manager is further operable to:select a first video bus from the one or more video buses; receive thesecond image from the content processor over the first video bus; andstore the second image in the memory medium; wherein, in selectivelyinserting a second image into the first image generated by the firstcomputing system for display on the first display device, the videomanager is operable to retrieve the second image from the memory mediumand insert the second image into the first image.
 29. A video manager,comprising: a video switch comprising: a memory medium; and a processorcoupled to the memory medium; and a communications processor coupled tothe video switch through one or more video buses; wherein the videoswitch is operable to couple to a plurality of co-located computingsystems, and to further couple to a plurality of human interfaces,wherein each of the human interfaces is located remotely from theco-located computing systems, wherein each of the human interfacesincludes a display device for displaying images, and wherein each of thehuman interfaces includes a camera for acquiring an image; wherein thecommunications processor is operable to receive a first image from acamera comprised in a respective first human interface; wherein thevideo switch is further operable to: receive a second image generated bya first computing system to be displayed on a first display device in arespective second human interface; receive the first image from thecommunications processor; and selectively insert the first image intothe second image generated by the first computing system for display onthe first display device.
 30. The video manager of claim 29, wherein thevideo manager is further operable to: select a first video bus from theone or more video buses; receive the first image from the communicationsprocessor over the first video bus; and store the first image in thememory medium; wherein, in selectively inserting a first image into thesecond image generated by the first computing system for display on thefirst display device, the video manager is operable to retrieve thefirst image from the memory medium and insert the first image into thesecond image.
 31. A videoconferencing system, comprising: a plurality ofcomputing systems, wherein the plurality of computing systems arelocated at a common location; a plurality of human interfaces, whereineach of the human interfaces is located remotely from the commonlocation, wherein each of the human interfaces includes a display devicefor displaying images and a camera for acquiring an image, and whereineach computing system is operable to generate a respective first imageto be displayed on a display device in a respective human interface; aplurality of communication mediums coupling each of the plurality ofcomputing systems to at least one of the plurality of human interfaces;and a video manager coupled to each of the plurality of computingsystems, wherein, for each human interface, the video manager isoperable to: receive the respective first image from a respective one ofthe plurality of computer systems; receive one or more respective secondimages from a respective one or more cameras of a respective one or morehuman interfaces of the plurality of human interfaces; selectivelyinsert the respective second images into the first image, therebygenerating a third image comprising at least a portion of the respectivefirst image and at least a portion of each of the respective secondimages; and transmit the third image to the human interface for displayon the display device.
 32. The system of claim 31, wherein the videomanager is further coupled to one or more external cameras over anetwork, wherein each of the one or more external cameras is operable toacquire a respective fourth image; and wherein the video manager isfurther operable to: receive one or more respective fourth imagesrespectively from the one or more external cameras; selectively insertthe one or more respective fourth images into the first image, therebygenerating the third image comprising at least a portion of therespective first image and at least a portion of each of the respectivesecond images, and at least a portion of each of the respective fourthimages; and transmit the third image to the human interface for displayon the display device.